Hybrid recording unit including portable video recorder and auxillary device

ABSTRACT

The invention enables “low attention recording,” i.e., recording that need not appreciably affect interaction of a recorder (i.e., a person in control of a recording unit according to the invention) or others with the environment in which the recording is taking place. The low attention recording enabled by the invention can be particularly advantageously employed in “social recording,” i.e., recording in which multiple participants in an event are recording the event. As described in more detail below, advantageous aspects of the invention can be provided by a small, lightweight, wearable recording unit. In particular, a “hybrid recording unit” having the above-described characteristics and advantages is constructed by adding to a portable video recorder (e.g., camcorder, portable dockable videotape recorder (VTR)) one or more devices (an “auxiliary device”) that provide additional functionality to the portable video recorder. The auxiliary device can advantageously provide, for example, one or more of the following capabilities: marking, position sensing, physiological monitoring and/or biometric identification.

This Application claims the benefit of Provisional Application No.60/114,808, filed Jan. 5, 1999. This is a continuation of applicationSer. No. 09/408,284, file Sep. 29, 1999, now abandoned.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to the following commonly owned, co-pendingUnited States Patent Applications, filed on the same date as the presentapplication, the disclosures of which are incorporated by referenceherein: “Low Attention Recording, with Particular Application to SocialRecording,” by Henry B. Strub et al., Ser. No. 09/408,481; and “LowAttention Recording Unit for Use by Vigorously Active Recorder,” byHenry B. Strub et al., Ser. No. 09/408,482.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates, in general, to recording an event and to devicesused to record an event. The invention further, and more particularly,relates to recording an event in which multiple persons participate (a“group event”) and, even more particularly, to recording of the event bymultiple participants (i.e., from multiple points of view), oftensimultaneously. Most particularly, the invention relates to a “hybridrecording unit” having capabilities useful in such recording that isconstructed by adding to a portable video recorder one or more devicesthat provide additional functionality to the portable video recorder.

2. Related Art

It is often desirable to record the content of an event by obtaining avisual record of the event. People often desire to capture a visualrecord of an event (particularly one in which they participate) as a wayof facilitating reminiscence of the event. Or, a person may desire tocommunicate the content of an event to one or more other people who werenot present. However, existing devices for obtaining a visual record ofan event have characteristics that can diminish their desirability astools for recording an event, particularly when the person operating therecording device desires to participate in the event.

A camera (“still camera”) for taking a still photograph is one exampleof a device that has been used to obtain a visual record of an event.However, still photographs can be deficient in representing the contentof an event. Some (usually, most) of the visual content of the event isnecessarily lost (e.g., the visual content that occurs between the timesthat photographs are taken). It is also difficult to capture interestingactivity that occurs spontaneously, since, absent fortuity, a photographwill not be taken during the occurrence—or, at least, at the beginningof the occurrence—of such activity. Further, the content that iscaptured lacks context and motion that can convey the nuance and/orvitality of the event. Additionally, a still camera does not recordaudio content. In addition to deficiencies in the recording of content,use of a still camera also typically requires sufficient attention fromthe operator of the camera (e.g., aiming of the camera and setting ofcamera control parameters) that the camera operator cannot participatein the event when a photograph is being taken. (Use of a tripod andtimer can eliminate this problem, but generally eliminates thespontaneity of the recorded content as well, and typically requiresrather elaborate setup that may not be compatible with the event or beconducive to maximum enjoyment of the event by either the cameraoperator or other participants in the event.) The camera operator can befurther distracted from participation in the event by the need to carrythe camera even when the camera is not in use. Additionally, in normaloperation of a still camera, the camera operator is not part of thephotograph. (This may be particularly undesirable when a primary purposeof the camera operator in obtaining the visual recording is tofacilitate later reminiscence of the event.) Further, the use of atypical still camera is readily apparent to other people in the vicinityof the camera operator; such awareness of the use of the camera maycause people to alter their activity (if they perceive that they arewithin the field of view of the camera) or otherwise interfere with thetaking of photographs that record the “natural” occurrence of the event.

A camcorder is another example of a device that has been used to obtaina visual record of an event. Since a camcorder can continuously recordan event, and since a camcorder can record audio content, a camcordercan, to some extent, overcome the above-noted deficiencies of a stillcamera in representing the content of an event. However, since thecamcorder operator must focus a considerable amount of attention onoperation of the camcorder, the camcorder operator is, to a significantextent, prevented from being an active participant in the event beingrecorded. Even if the camcorder is not being used at particular time torecord the event, carrying the camcorder can prevent or significantlyimpede participation in the event. Moreover, the inconvenience of usinga camcorder may generally deter the camcorder operator from using thecamcorder, thus tending to reduce the number of events that are recordedor the amount of a particular event that is recorded. Additionally, thecamcorder operator is, excepting atypical operation of the camcorder,not included in the visual recording of the event. Further, camcordersare constructed so that their use is readily apparent to others; suchawareness may result in a change in behavior of people within and/ornear the field of view of the camcorder so that the content of therecording is artificially altered from the content that would haveoccurred had the event proceeded naturally (i.e., without awareness byparticipants in the event of the recording). Typical camcorders alsohave physical deficiencies which can inhibit their operation. Forexample, battery capacity is often limited to an extent that undesirablylimits the duration of time that the camcorder can operate withoutinterruption. Consumer camcorders also typically are constructed suchthat the quality of the audio recording obtained may be deficient: themicrophone(s) commonly pick up background noise associated withoperation of the camcorder motor and are typically positioned at asingle location so that sound is sensed from only one point of view.

Often, an event that it is desirable to visually record is a “groupevent” (i.e., an event observed, or participated in, by multiplepersons). Often, some or all of the people that are part of a groupevent are associated by a desire to share the experience of the eventtogether. A group hike is an example of such an event. A family's tripto an amusement park is another example of such an event. Such groupevents are frequently events that it is desired to record in order tofacilitate later reminiscence. Further, such group events are also oftenevents in which the associated people desire to participate.

Recording of a group event using a still camera or camcorder suffersfrom the deficiencies (e.g., inhibition of participation in the event bythe operator of the recording device, as well as lack of presence of theoperator in the recording of the event) described above. However, asidefrom those deficiencies, recording of a group event presents additionalchallenges.

For example, the use of a single recording device—of any type—to recorda group event limits the content of the recording, since the recordingcan only be obtained from a single point of view. When a camcorder isused to record an event, it is often the case that a single camcorder isused. (This is especially so because camcorders are so cumbersome touse.) Further, though it may be more likely that multiple (or even all)participants in a group event carry a still camera, typically only oneparticipant takes a photograph at any given time, thus providing only asingle point of view from which the event is recorded at a particulartime.

Additionally, even if multiple devices are used to record a group event,still cameras and camcorders suffer from other limitations. For example,existing such devices do not provide for sharing of recordings betweendevices during the event, thus inhibiting or preventing coordination ofrecording by multiple devices so as to best capture the content of theevent (e.g., so as to increase the amount of different content recorded,increase the number of participants recorded and/or record interestingcontent from multiple viewpoints). Additionally, existing consumerdevices do not provide for synchronization in time or coordination inspace of the recordings obtained by the devices, making it difficult tolater present the recordings together in a coherent fashion. It istypically desirable to acquire a visual recording at a distance from thecontent being recorded (so as to include more visual content), while itis typically desirable to acquire an audio recording near the contentbeing recorded (so as to make it easier and less expensive to recordproximate audio content), a conflict that might potentially be overcomeby mixing of multiple recordings of an event obtained at differentlocations by different recording devices. However, the inability tocoordinate and synchronize recordings obtained by multiple still camerasand/or camcorders inhibits the use of those multiple recordings in amanner that mitigates the above-described tension between visual andaudio recording (e.g., by mixing a visual recording of content obtainedat a distance by one recording device with an audio recording acquirednear that content by a different recording device).

During recording of an event, it can be desirable to mark the recording.Some recording devices (e.g., some camcorders) can enable a mark of asingle type to be associated with recording data at the time therecording data is obtained, the mark intended to indicate that thecontent represented by the recording data is of interest and may beparticularly desirable to view during later display of the recording.The use of such a mark can enable the medium on which the recording datais stored to be searched to identify and display one or more parts of arecording in accordance with the location of mark(s) within therecording.

In a camcorder, marking is most easily implemented by storing markingdata on the videotape onto which recording data is recorded. However,storing marking data on a videotape (or other linear data storagemedium) has disadvantages. For example, searching for marks can take anundesirably long time, because the entire recording between a currentlocation within the recording and a marked location within the recordingmust be traversed to reach the marked location. Additionally, for asimilar reason, moving from a display of one marked part of a recordingto another marked part of a recording can take an undesirably long time.

It is desirable to have greater flexibility in marking a recording. Forexample, it is desirable to be able to mark recording data of arecording other than the recording data being obtained at the time themark is specified. This is particularly so since it may not be realizedthat it is desirable to mark a part of a recording until some time otherthan the time at which that part of the recording is being obtained. Itis also desirable to be able to mark a recording with marks of differenttypes, each type of mark having a different meaning. Such marking couldbe used to better facilitate searching and/or display of the recordingby enabling increasing specificity in the identification of the contentof parts of the recording. Further, it is desirable to store recordingdata and marking data on a non-linear data storage medium, since thatenables faster searching for occurrence of a mark in a recording andalso enables more flexible movement between different marked parts of arecording.

SUMMARY OF THE INVENTION

The invention enables “low attention recording,” i.e., recording thatneed not appreciably affect interaction of a recorder (i.e., a personhaving the capability of controlling a recording unit according to theinvention) or others with the environment in which the recording istaking place. The low attention recording enabled by the invention canbe particularly advantageously employed in “social recording,” i.e.,recording in which multiple participants in an event are recording theevent. As described in more detail below, advantageous aspects of theinvention can be provided by a small, lightweight, wearable recordingunit.

The invention facilitates low attention recording in a variety of ways.In particular, the invention allows a recorder to meaningfullyparticipate in the event while recording the event. For example, as isclearer from the detailed description of the invention below, arecording unit according to the invention can be operated with little orno attention from the recorder (i.e., beginning operation of therecording unit can be accomplished with little effort, and, once turnedon to begin recording, operation of the recording unit may not requirefurther interaction from the recorder), the recording unit can beadapted so that the recording unit does not physically interfere (e.g.,obstruct vision or movement) with the recorder's participation in theevent, and the recording unit can be adapted so that the physicalcharacteristics of the recording unit do not distract the recorder(e.g., the recording unit is not too heavy or bulky). A recording unitaccording to the invention can also be implemented so that recording ofthe event by the recording unit does not affect participation in theevent by other participants to a significant extent. For example, asdescribed in more detail below, a recording unit according to theinvention can be adapted to have a relatively inconspicuous physicalappearance and to operate in a relatively inconspicuous fashion.

The invention also facilitates social recording in a variety of ways.For example, a recording unit according to the invention can be adaptedto enable recordings to be shared between or among recording unitsduring the event. A recording unit according to the invention can alsobe adapted to enable recorders to communicate with each other during theevent.

The invention can also enable a recording to be marked as an event isbeing recorded. Marking capability in accordance with the invention canbe implemented in a variety of ways. A mark can be specified using atactile control mechanism (which can be physically integrated with, orseparate from, other components of the recording unit) adapted for thatpurpose or can be specified by a voice command. A mark can also bespecified based upon analysis of audio recording data, visual recordingdata or both. A mark can also be specified, or an existing marksupplemented or modified, based on data other than recording data (e.g.,physiological monitoring data, biometric data, position data) that isacquired by the recording unit. Advantageously, the invention can enablethe use of multiple types of marks, each type of mark having aparticular meaning. The invention can also advantageously enable thespecification of a mark so that the mark is associated with recordingdata obtained at a time other than the time at which the mark isspecified, i.e., prior to the time of the mark (“predictive marking”) orafter the time of the mark (“retrospective marking”). Restropectivemarking is especially consonant with the goal of unobtrusive, lowattention recording, since, often, a recorder who is engaged inparticipating in the event will not appreciate the desirability ofmarking particular recording data until some time after activity ofinterest has occurred. Additionally, the invention can enable markingcapability to be implemented so that specification of a mark causespredetermined operation of the recording unit in accordance with thetype of the mark (“augmented marking”).

In particular, a recording unit (a “hybrid recording unit”) according tothe invention having the above-described characteristics and advantagesis constructed by adding to a portable video recorder (e.g., camcorder,portable dockable videotape recorder (VTR)) one or more devices (an“auxiliary device”) that provide additional functionality to theportable video recorder. The auxiliary device can advantageouslyprovide, for example, one or more of the following capabilities:marking, position sensing, physiological monitoring and/or biometricidentification.

In one embodiment of the invention, a hybrid recording unit for use inrecording an event includes: i) a portable video recorder for acquiringrecording data representing the event, the portable video recorderincluding an external communication port for communicating with anexternal device; ii) an auxiliary device for producing and/or acquiringauxiliary data during the recording of the event, the auxiliary deviceincluding means for communicating with the portable video recorder viathe external communication port of the portable video recorder (theauxiliary device also typically includes a non-volatile data storagedevice for storing auxiliary data, such as an EEPROM or flash memorydevice); and iii) means for establishing a temporal correspondencebetween the auxiliary data and recording data. The auxiliary device canadvantageously be implemented to acquire auxiliary data that is of atype that cannot be acquired by the portable video recorder.

For example, the auxiliary device can include marking apparatus forspecifying one or more marks that are associated with particularrecording data acquired by the portable video recorder. Or, for example,the auxiliary device can include apparatus for sensing the position ofthe hybrid recording unit (e.g., a GPS receiver; a transmitter orreceiver for transmitting or receiving, respectively, a signal to areceiver or from a transmitter, respectively, positioned at a knownlocation or on a known device; an electronic compass; a tilt sensor; aninertial reference system). The auxiliary device can also includeapparatus for acquiring physiological data representing a physicalcharacteristic of a recorder (e.g., one or more of a heart ratemonitoring device, a galvanic skin response measurement device, an EEGmonitoring device, an ECG monitoring device, a force sensor, an EMGmonitoring device, a body temperature measuring device and a respirationrate measurement device). The auxiliary device can also include abiometric device.

The portable video recorder and auxiliary device can communicate witheach other using any of a variety of communication protocols, such as,for example, the LANC protocol, the JLIP protocol, an IEEE 1394protocol, or a “5-pin editing terminal” protocol. Further, communicationbetween the portable video recorder and auxiliary device can beimplemented using wireless and/or wired methods and apparatus.

The temporal correspondence between the auxiliary data and recordingdata can be established using a method including the steps of: i)causing a synchronization event to occur prior to, or at the time of,beginning recording of the event; ii) producing synchronization data,from the synchronization event, that identifies the recording position,at the beginning of the recording, of a first non-volatile data storagedevice on which recording data is stored and/or a second non-volatiledata storage device on which auxiliary data is stored; and iii) storingthe synchronization data. The invention can be implemented so that themethod of establishing a temporal correspondence between the auxiliarydata and recording data can be performed automatically wheneverrecording is begun. Further, the invention can be implemented so thatthe method of establishing a temporal correspondence between theauxiliary data and recording data is not performed if, at the beginningof recording, the recording position of the first non-volatile datastorage device at which it is desired to begin storing recording data isa beginning data storage position of the first non-volatile data storagedevice.

In a further embodiment of the foregoing method, which is particularlyapplicable when the first non-volatile data storage device is a lineardata storage medium (e.g., videotape), the step of causing asynchronization event can further include the steps of: i) identifying afirst value of a data counter of the first non-volatile data storagedevice on which the recording data is stored, the first data countervalue corresponding to a recording position of the first non-volatiledata storage device at which it is desired to begin storing recordingdata; ii) operating the first non-volatile data storage device so thatthe recording position of the first non-volatile data storage devicecorresponds to a beginning data storage position of the firstnon-volatile data storage device; iii) identifying a second value of thedata counter corresponding to the beginning data storage position; andiv) operating the first non-volatile data storage device so that therecording position of the first non-volatile data storage device againcorresponds to the recording position at which it is desired to beginstoring recording data. Further, in such an embodiment, the step ofproducing synchronization data can further include the step ofdetermining a data counter offset by comparing the first and second datacounter values, the data counter offset identifying the recordingposition of the first non-volatile data storage device at the beginningof the recording, and the step of storing the synchronization data canfurther include the step of storing the tape counter offset. In a stillfurther embodiment of the foregoing method, the step of producingsynchronization data further includes the step of providing one or moreincremental data counter values during recording, each incremental datacounter value being associated with recording data and auxiliary dataobtained at the same time, and the step of storing the synchronizationdata further includes the step of storing the one or more incrementaldata counter values.

In another further embodiment of the general method described above ofestablishing a temporal correspondence between the auxiliary data andrecording data, the step of causing a synchronization event furtherincludes the step of producing a visual and/or aural phenomenon at thebeginning of recording, the step of producing synchronization datafurther includes the step of operating the first data acquisition deviceso as to acquire recording data representing the visual and/or auralphenomenon, and the step of storing the synchronization data furtherincludes the step of storing the recording data representing the visualand/or aural phenomenon. The visual and/or aural phenomenon can be, forexample, a predefined sound or a flash of light.

In still another further embodiment of the general method describedabove of establishing a temporal correspondence between the auxiliarydata and recording data, in which the first non-volatile data storagedevice is a videotape, the step of causing a synchronization eventfurther includes the steps of: i) determining whether a time code isstored at the current location of the videotape; ii) if a time code isnot stored at the current location of the videotape, rewinding thevideotape to a location at which storage of a time code on the videotapeis detected; and iii) if a time code is stored at the current locationof the videotape, winding the videotape to a location at which storageof a time code on the videotape is not detected, then rewinding thevideotape to a location at which storage of a time code on the videotapeis detected.

In another embodiment of the invention, an auxiliary device for use witha portable video recorder adapted to acquire recording data representingan event includes: i) apparatus for producing and/or acquiring auxiliarydata during recording of the event; ii) apparatus for communicating withthe portable video recorder via an external communication port of theportable video recorder; and iii) a non-volatile data storage device forstoring auxiliary data. The communication between the auxiliary deviceand portable video recorder can advantageously be implemented usingwireless methods and apparatus.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates use of recording units according to the invention.

FIG. 2 is a block diagram illustrating the functional components of arecording unit in accordance with the invention.

FIG. 3 is a block diagram illustrating the functional components ofanother recording unit in accordance with the invention, the recordingunit having enhanced capabilities as compared to the recording unit ofFIG. 2.

FIG. 4 is a perspective view of a visual data acquisition device of arecording unit according to the invention attached to a head harnessthat is worn by a recorder.

FIG. 5 illustrates a control interface device according to an embodimentof the invention.

FIG. 6 illustrates another way of packaging a recording unit accordingto the invention.

FIG. 7 is a block diagram of the functional components of a recordingunit according to a particular embodiment of the invention that isparticularly well suited for use in recording an event in which therecorder engages in vigorous physical activity.

FIGS. 8A-8C are perspective views of a recording unit representing aparticular physical instantiation of the recording unit of FIG. 7, therecording unit illustrated in FIGS. 8A-8C as worn by a recorder.

FIGS. 9A and 9B are perspective views of a recording unit representinganother particular physical instantiation of the recording unit of FIG.7. FIG. 9A is a front perspective view of the recording unit being wornby a recorder. FIG. 9B is a side perspective view when the recordingunit is not being worn by a recorder.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 illustrates use of recording units according to the invention. InFIG. 1, four people are participating in, or observing, an event. (Forconvenience, herein, all such people are referred to as “participants.”)Each of the participants (designated by the numerals 101, 102, 103 and104) are equipped with a recording unit (designated by the numerals 105,106, 107 and 108). Each of the recording units 105, 106, 107 and 108, inaccordance with the invention, enables the corresponding participant(recorder) to obtain a recording of the event in a manner that does notinterfere with participation in, or observation of, the event, either bythe recorder or other participants. (Generally, a “recorder” is anyperson having the capability of controlling a recording unit accordingto the invention.) Additionally, in the use of the invention shown inFIG. 1, since there are multiple recorders, the event can besimultaneously recorded from multiple points of view. Further, theinvention enables the above-described recording from points of view thatare “in” the event, since the recording is done by participants in theevent. Thus, the invention can enable first person recording, frommultiple points of view simultaneously, while minimizing or eliminatingthe effect of the act of recording upon the conduct of the event.

The type of recording enabled by a recording unit according to theinvention is sometimes referred to herein as “low attention recording,”the term indicating the salient feature of a recording unit according tothe invention, which is that the act of recording need not—and typicallydoes not—(appreciably) affect interaction of the recorder or others withthe environment in which the recording is taking place. Recording inwhich multiple participants in an event are recording the event issometimes referred to herein as “social recording.” As will be morecompletely appreciated from the description of the invention herein, thelow attention recording enabled by a recording unit according to theinvention can be particularly advantageously employed in socialrecording situations.

A recording unit according to the invention is adapted to obtain visualdata representing a visual record of the event. Visual data that can beobtained by a recording unit according to the invention can include dataobtained by any device for obtaining a visual recording, such as, forexample, a video camera, motion picture camera or a still image camera.Typically, a recording unit according to the invention is also adaptedto obtain audio data representing an aural record of the event. (Herein,such visual and/or audio data is sometimes referred to as “recordingdata.”) A recording unit according to the invention can also be adaptedto obtain and/or produce other data representing a record of some aspectof the event. (Herein, data other than recording data that is obtainedand/or produced by a recording unit according to the invention issometimes referred to as “augmenting data.”) For example, a recordingunit according to the invention can be adapted to produce marking data.A recording unit according to the invention can also be adapted toobtain data regarding the position of the recording unit. A recordingunit according to the invention can also be adapted to recordphysiological data (e.g., heart rate, galvanic skin response):physiological data may be useful to enable the emotional state of therecorder (e.g., excitement, boredom) during recording to be inferred.Additionally, a recording unit according to the invention can be adaptedto obtain (and, perhaps, store) biometric data (e.g., fingerprint,retinal pattern).

As indicated above, a particularly advantageous aspect of the inventionis that a recording unit can be made to be relatively unobtrusive. Oneaspect of the unobtrusiveness of a recording unit according to theinvention is that the recorder is able to meaningfully participate inthe event while recording the event. For example, a recording unitaccording to the invention is preferably adapted so that the operationof the recording unit requires little or no attention from the recorderduring recording (i.e., beginning operation of the recording unit can beaccomplished with little effort, and, once turned on to begin recording,operation of the recording unit may not require further interaction fromthe recorder). A recording unit according to the invention is alsoadapted so that the recording unit does not physically interfere (e.g.,obstruct vision or movement) with the recorder's participation in theevent. Preferably, a recording unit according to the invention is alsoadapted so that the physical characteristics of the recording unit donot distract the recorder (e.g., the recording unit is not too heavy orbulky). Another aspect of the unobtrusiveness of a recording unitaccording to the invention is that participation in the event by otherparticipants is not affected to a significant extent by the recording ofthe event. For example, a recording unit according to the invention canbe adapted to have a relatively inconspicuous physical appearance and tooperate in a relatively inconspicuous fashion, thus minimizing oreliminating inhibitions that participants in the event may otherwisefeel if the recording unit is a conspicuous device that provides anongoing visceral reminder that their actions are being recorded.

A recording unit according to the invention can include capabilitiesthat facilitate low attention recording. For example, a recording unitaccording to the invention can be adapted to enable display in real timeof the recording being obtained by the recording unit. This enables therecorder to monitor, with little effort, the content of the recording asthe recording is obtained so that the recorder can, if desired, takeappropriate action to ensure that desired content is obtained.Additionally, viewing a recording display can increase a recorder'sconfidence that the recording unit is operating properly, making iteasier for the recorder to ignore the recording unit and concentrate onparticipating in the event. It is important to note, however, that suchmonitoring of the recording need not necessarily occur; as indicatedabove, the recording units can be (and preferably are) constructed sothat recordings with useful content can be obtained with littleinteraction by the recorder with the recording unit.

A recording unit according to the invention can also be adapted toenable the recorder to mark the recording as the event is beingrecorded. (“Marking” is used herein broadly to indicate any annotationof the recording that is made in response to an action by the recorderintended to create such annotation.) The capability of marking therecording enables the recorder to, with little effort, augment therecording in a way that facilitates and/or enhances use (e.g.,searching, display and/or processing) of the recording, either during orafter recording the event. There are many ways in which markingcapability can be implemented in a recording unit according to theinvention. For example, in some embodiments of the invention, asincidents of particular interest occur during the event being recorded,a dedicated marking mechanism can be used to specify a mark that isassociated with the recording data being obtained at that time. (Asdescribed in more detail below, a mark may have a unique identificationthat, for example, identifies content in the marked part of therecording and/or the identity of the person who specified the mark.)Narrative description (either textual description entered via anappropriate input device or spoken description acquired by anappropriate audio data acquisition device) of an incident can be addedto a recording as mark(s) or part(s) of mark(s). A mark can be specifiedusing a tactile control mechanism adapted for that purpose. A mark canalso be specified by voice commands. Advantageously, a recording unitaccording to the invention can enable the use of multiple types ofmarks, each type of mark having a particular meaning. A recording unitaccording to the invention can also advantageously enable thespecification of a mark so that the mark is associated with recordingdata obtained at a time other than the time at which the mark isspecified (“non-contemporaneous marking”). Non-contemporaneous markingcan be implemented to enable the recorder to mark recording dataobtained prior to the time of the mark (“retrospective marking”) orsubsequent to the time of the mark (“predictive marking”).Non-contemporaneous marking is especially consonant with the goal ofunobtrusive, low attention recording by the recording unit, since,often, a recorder who is engaged in participating in the event will notappreciate the desirability of marking particular recording data untilsome time after activity of interest has occurred.

A recording unit according to the invention can also includecapabilities that facilitate social recording. For example, a recordingunit according to the invention can be adapted to enable recordings tobe shared between or among recording units during the event so thatrecorders can observe recordings that have been, or are being, obtainedby other recording units. Again, this can, if a recorder is so inclined,aid the recorder in obtaining desired content, since the recorder can beaware of the content obtained by other recording units and can attemptto operate the recorder's recording unit so as to obtain a recordingthat is coordinated in some manner with the recording(s) being obtainedby other recording unit(s). Particularly if the recording unitsaccording to the invention also include the capability for recorders tocommunicate with each other during recording, this capability enablescollaborative social recording without need for the recorders to becomeoverly disengaged from participating in the event. (However, asindicated above, the invention contemplates that a recording unit can beemployed to obtain useful content without such monitoring of otherrecordings by the recorder.) Sharing of recordings during the event canalso be desirable because viewing the recording(s) of other recorder(s)can make the event more enjoyable.

A recording unit according to the invention can also be adapted so thatthe recording can be obtained from novel and/or interesting points ofview. As discussed in more detail below, this can be accomplished as aresult of adapting the recording unit so that the recording unit can bemounted on the recorder. A recording unit can also be adapted so thatthe recording can be obtained from any of multiple points of view. Asalso discussed more below, this can be accomplished by adapting therecording unit so that the recording unit can be mounted at differentlocations on the recorder.

Below, aspects of the invention are described in more detail, includingapparatus with which a recording unit according to the invention can beembodied, operation of a recording unit according to the invention,interaction of a recording unit according to the invention with otherrecording units during recording of an event, and post-event use of therecordings obtained by recording unit(s) according to the invention.

I. Recording an Event

A. Overview of Recording Unit Apparatus and Operation

As indicated above, it is desirable that a recording unit according tothe invention can be implemented so that the recording unit requireslittle attention from the recorder during recording, does not physicallyinterfere with, or distract from, the recorder's participation in theevent, and does not unduly occupy attention of other participants in theevent. Thus, for example, it is generally desirable that the recordingunit be relatively small and that the recording unit be relativelylightweight. Further, it is highly desirable that the recording unit beadapted to be mounted on (e.g., attached on, or worn by), rather thanheld by, the recorder. The recording unit should be mounted, andcomponents of the recording unit interconnected, so that the recordingunit does not obstruct the recorder's vision, hearing or motion, ordistract the recorder from participation in the event. The recordingunit should be capable of being operated easily: for example, it isdesirable that starting and stopping recording require littleinteraction by the recorder with the recording unit, and that therecording unit be adapted so that any required recording adjustments(e.g., focusing, brightness control) are made automatically by therecording unit as recording occurs. It is also desirable that therecording unit be capable of operation for a long period of time (e.g.,several hours) relative to the expected duration of activities duringwhich it is desired to use the recording unit: for example, a powersupply used with the recording unit should be capable of storingsufficient power to enable a desired duration of operation without needto recharge or replace the power supply.

FIG. 2 is a block diagram illustrating the functional components of arecording unit 200 in accordance with the invention. The recording unit200 includes the basic capabilities of a recording unit for use in asystem according to the invention.

The recording unit 200 includes a system controller 201, a dataacquisition device 202, a data storage device 203 and a controlinterface device 204. (In practice, each of the devices of the recordingunit 200—as well as of the recording unit 300, described below withrespect to FIG. 3, or any other recording unit according to theinvention—can be embodied by a single apparatus or multiple physicallydistinct apparatus, and can also be embodied in such apparatus togetherwith one or more other devices of the recording unit.) The systemcontroller 201 controls the operation of, and mediates interactionamong, the other components of the recording unit 200. The dataacquisition device 202 acquires recording data representing the contentof the event being recorded. The recording data acquired by the dataacquisition device 202 is stored in the data storage device 203 for usesubsequent to the event in observing the recording of the event. Thecontrol interface device 204 enables the recorder to control theoperation of the recording unit 200 and can also provide information tothe recorder regarding the operation of the recording unit 200.

FIG. 3 is a block diagram illustrating the functional components ofanother recording unit 300 in accordance with the invention. Therecording unit 300 has enhanced capabilities as compared to therecording unit 200 described above with respect to FIG. 2. It is to beunderstood that there are many other possible embodiments of a recordingunit according to the invention in addition to those shown in FIGS. 2and 3: such embodiments may include only some of the components of therecording unit 300 that are present in addition to the components of therecording unit 200, and can also include other components not shown inFIGS. 2 and 3. FIGS. 2 and 3 are merely illustrative of the types ofcomponents that can be present in a recording unit according to theinvention.

The recording unit 300 includes a system controller 301, a visual dataacquisition device 302, an audio data acquisition device 303, a dataprocessing device 304, a data storage device 305, a visual recordingdisplay device 306, an; audio recording display device 307, a controlinterface device 308, a transmitter 309, a receiver 310 and a positionsensing device 311. As in the recording unit 200, described above, thesystem controller 301 controls the operation of, and mediatesinteraction among, the other components of the recording unit 300. Thevisual data acquisition device 302 and audio data acquisition device 303acquire visual and audio recording data, respectively, representing thecontent of the event being recorded. Depending upon the particularimplementation of the recording unit 300, the acquired recording datacan be stored directly in the data storage device 305, or can first beprocessed (e.g., compressed) in the data processing device 304, thenstored in the data storage device 305. The visual recording displaydevice 306 and the audio recording display device 307 enablepresentation to the recorder of a display (which is typically areal-time display) of the visual recording and audio recording,respectively, being obtained by a compatible recording unit (which canbe the recording unit 300 or another recording unit). As in therecording unit 200, the control interface device 308 enables therecorder to effect operation of the recording unit 300 and can alsoprovide information to the recorder regarding the operation of therecording unit 300. The transmitter 309 transmits signals (“recordingdata signals”) representing recording data acquired by the recordingunit 300, so that such recording data can be used by other recordingunits (e.g., to generate a display of the visual and/or audio recordingbeing obtained by the recording unit 300, or to enable processing ofrecording data obtained by the recording unit 300 for use in affectingthe manner in which another recording unit records, processes or storesrecording data). The receiver 310 receives recording data signalsrepresenting recording data acquired by one or more other recordingunits to enable generation of a display of the recording being obtainedby those recording units. (The receiver 310 can also receive recordingdata signals transmitted by the transmitter 309 and use such signals togenerate a display of the recording being obtained by the recording unit300.) The position sensing device 311 can be used to ascertain theposition (location and/or orientation) of the recording unit 300, theposition information being used, for example, to augment the display ofthe recording or to facilitate integration of the recording obtained bythe recording unit 300 with the recording(s) obtained by other recordingunit(s).

Particular apparatus that can be used to embody the components of therecording units 200 and 300, as well as additional aspects of theoperation of recording units like the recording units 200 and 300, aredescribed in more detail below.

B. Recording Unit Components

1. Power Supply

For typical implementations of a recording unit according to theinvention, a portable power supply (e.g., a battery) must be included aspart of the recording unit. (However, other implementations arepossible: for example, power may be transmitted—preferably, in awireless manner—from a fixed power supply that is separate from therecorder, so that the recording unit receives adequate power foroperation so long as the recording unit remains within a defined area.)In general, it is desirable that such a portable power supply be capableof storing sufficient energy to enable operation of the recording unit,without recharging or replacing the power supply, for a period of timelong enough to encompass the expected duration of events with which therecording unit is intended to be used. To enable a particular powersupply to operate a recording unit for a desired period of time withoutrecharging or replacement, it is, in general, desirable that thecomponents of the recording unit described below be implemented usingdevices that consume relatively little power.

An increase in the energy storage capacity of a power supply typicallyresults in an increase in the size of the power supply. Since it isgenerally desirable that the recording unit be small, the implementationof a power supply for use with a recording unit according to theinvention typically involves a trade-off between size and power storagecapacity. It is desirable that the power supply be capable of supplyingpower to enable operation of the recording unit for at least severalhours (e.g., 2-5 hours) without recharging or replacing the powersupply. Thus, the power supply is preferably selected, in view of thepower consumption characteristics of the other components of therecording unit, so as to enable such operation.

It may be desirable to implement a recording unit according to theinvention so that the power supply is integrated with anotherperiodically replaceable part of the recording unit (e.g., anon-volatile data storage device for storing recording data), therebyenabling both components to be conveniently replaced at the same time.In such case, the characteristics of the components (e.g., energystorage capacity of the power supply and data storage capacity of thenon-volatile data storage device) are preferably established so that thecomponents will typically be ready for replacement at about the sametime.

2. System Controller

In general, as indicated above, a system controller of a recording unitcan be any device that controls the operation of other components of therecording unit and mediates interaction among those components to effectperformance of the functions of the recording unit in response to(either directly or indirectly) input from the recorder. Theconstruction and operation of a system controller for use in a recordingunit according to the invention can be implemented by those skilled inthe art in view of the description herein of other components andaspects of a recording unit according to the invention.

The system controller can also perform functions in addition to theabove-described control and mediation. Preferably, the system controlleris operated to store information that identifies particular storedrecording data, in some manner. For example, the system controller caninclude a clock that is used to time-stamp the recording data obtainedby the recording unit. (The time-stamps can be used, for example, todisplay the time at which a displayed recording was obtained, to locateparticular stored recording data, or to enable post-eventsynchronization of the display of recordings obtained by differentrecording units.) The system controller can also cause information to bestored that identifies the recording unit (and/or recorder) thatobtained the recording data, and/or that identifies the position of therecording unit at the time the recording data was obtained.Additionally, as described in more detail below, the system controllercan cause data to be stored that represents a mark specified by therecorder, the mark identifying the corresponding recording data in apredefined manner.

In general, the system controller can be embodied by any device ordevices that can be operated to accomplish the necessary functions, suchas a microcontroller or other processing device. The system controlleroperates in accordance with appropriately specified instructions and/ordata (e.g., computer programs) that are stored on a data storage deviceof the recording unit. Illustratively, a system controller for use in arecording unit according to the invention can be embodied by a Motorola68HC11 microcontroller (as is commercially available) appropriatelyprogrammed, as can readily be done those skilled in the art, toaccomplish the desired functions of the system controller, as describedherein.

3. Data Acquisition

a. Visual Data Acquisition

Generally, any visual data acquisition device (i.e., an image capturedevice) can be used in a recording unit according to the invention. Itis anticipated that use of a device that acquires a continuous stream ofvisual data (e.g., frames of visual data acquired at video frame rates,such as approximately 30 frames per second, as known by those skilled inthe art), such as a video camera, will be preferable for manyapplications of the invention. However, a still image camera can also beused.

It is desirable that the visual data acquisition device be capable ofacquiring high resolution images. However, the cost of a visual dataacquisition device typically increases as the resolution of the visualdata acquisition device increases. The visual data acquisition devicecan be chosen in view of these competing considerations. However, asdescribed in more detail below, it can be possible to process visualrecording data to enhance the quality of the images that the visualrecording data represents. If it is contemplated that such processingwill be performed during post-event use of the recording, it may bedesirable to use a relatively lower resolution and less expensive visualdata acquisition device with a recording unit according to theinvention.

Either an analog visual data acquisition device (e.g., an analog videocamera) or a digital visual data acquisition device (e.g., a digitalvideo camera) can be used. A digital visual data acquisition device canbe advantageous because then there is no need to perform conversion ofan analog visual recording to digital form (which can result in reducedimage quality and which necessitates additional apparatus) to enableprocessing of the visual recording data. Additionally, a digital visualdata acquisition device can typically enable better automatic devicecontrol: aspects of the recording process can be controlled better thanwith an analog visual data acquisition device, and more aspects of therecording process can be controlled. When the automatic control featuresof an analog visual data acquisition device fail to adequatelycompensate for changing conditions, the quality of the resultantrecording can be degraded quite noticeably. An analog visual dataacquisition device can be advantageous because such devices aretypically cheaper, more widely available, more generally familiar, andavailable in a wider variety than digital visual data acquisitiondevices.

The size and weight of the visual data acquisition device areconstrained by the need to make the visual data acquisition deviceunobtrusive (so that the visual data acquisition device is notdistracting to the recorder of other people in the vicinity of therecorder), capable of being used in a manner that requires littleattention from the recorder (e.g., as indicated above, worn by therecorder), capable of being used in a manner that does not interferewith the recorder's participation in the event, and (if appropriate)comfortable for the recorder to wear. Generally, these considerationsmilitate in favor of making the visual data acquisition device as smalland lightweight as possible. For example, it can be desirable that thevisual data acquisition device weigh no more than several ounces. Theabove considerations can also affect the shape (to the extent that theshape of the visual data acquisition device can be chosen) of the visualdata acquisition device, as well as the location of the visual dataacquisition device with respect to the recorder's body (when the visualdata acquisition device is worn), as discussed further below.

In keeping with the goal of minimizing the need for interaction by therecorder with the recording unit, the visual data acquisition device ispreferably adapted to be mounted on (e.g., attached on or worn by, asopposed to held by) the recorder. The visual data acquisition deviceshould be stably mounted on the recorder to minimize or eliminateblurring of the visual recording. However, if the visual dataacquisition device is sufficiently small (as is generally desirable forother reasons, as discussed elsewhere herein), mounting the visual dataacquisition device so that adequate stability (with respect to the partof the body on which the visual data acquisition device is mounted) isobtained is typically not a problem.

Not only should the visual data acquisition device be stably mounted onthe recorder, but the visual data acquisition device should be mountedon a part of the body of the recorder that remains relatively stableduring the event. Thus, it is usually not desirable to mount the visualdata acquisition device on the hand or part of the arm distant from theshoulder, since those parts of the body will often experience swiftand/or far ranging motion. For the same reason, it is usually notdesirable to mount the visual data acquisition device on the leg distantfrom the hip. Adequate stability of the visual data acquisition devicecan typically be obtained by mounting the visual data acquisition deviceon, for example, the waist, chest, shoulder or head of the recorder.

However, in some situations, it may not be possible for the visual dataacquisition device to be held sufficiently still to enable a highquality recording to be obtained (e.g., if the recorder is running orvigorously hiking). As discussed in more detail below, it can bepossible to process the visual recording data after it is obtained toenhance the quality (e.g., remove some of the jerkiness) of portions ofa visual recording that have been obtained when the visual dataacquisition device was unstable. Additionally, the mechanism formounting the visual data acquisition device on the recorder can beimplemented with an active control mechanism that, in response tomovement of the visual data acquisition device, appropriately adjuststhe position of the visual data acquisition device to maintain a stablefilming direction.

The location on the recorder at which the visual data acquisition deviceis mounted can also be affected by consideration of the desiredcharacteristics of the point of view of the visual recording. In thisvein, the height of the visual data acquisition device and the directionin which the visual data acquisition device records must be considered.For example, it is anticipated that it will often be desirable to recordfrom a height at or above chest level. It is also anticipated that,typically, it will be desirable to position the visual data acquisitiondevice on a front part of the recorder's body, so that the visual dataacquisition device records activity that occurs in front of therecorder, the assumption being that the recorder will usually positionhis body so that it faces the activity of interest to the recorder.Further, it may be desirable to mount the visual data acquisition deviceon the front of the recorder's head (e.g., on the forehead), since it iseven more likely that the recorder will position his head to look in thedirection of activity of interest to the recorder. Mounting the visualdata acquisition device on the head also enables the point of view ofthe visual recording to be changed without interaction by the recorder,and enables changes in point of view that are expected, for the reasongiven above, to increase the likelihood of obtaining content ofinterest. FIG. 4 is a perspective view of a visual data acquisitiondevice 401 attached to a head harness 402 that is worn by a recorder403, illustrating one way of mounting a visual data acquisition deviceon the front of the recorder's head. However, mounting the visual dataacquisition device on the recorder's head can make the visual dataacquisition device more visible (and thus, perhaps, more distracting ordisconcerting) to others. Since one goal of the invention is to enableunobtrusive recording, this can be undesirable, as discussed elsewhereherein. The visibility of the visual data acquisition device can bereduced (and the associated distractiveness mitigated) by mounting thevisual data acquisition device on the side of the recorder's head (by,for example, mounting the visual data acquisition device on a headbandnear the recorder's ear, or on the side of a pair of eyeglasses worn bythe recorder), rather than on the recorder's forehead.

It may be particularly advantageous to implement the recording unit sothat the visual data acquisition device can be mounted at differentlocations on the recorder (“location-adjustable mounting”), thusenabling a recorder to choose to obtain the visual recording fromdifferent points of view for different events (or even to change thepoint of view of the visual recording during an event), as well asenabling different recorders (of, for example, different heights) toobtain a visual recording from the same point of view.Location-adjustable mounting of the visual data acquisition device canbe accomplished in any suitable manner.

For example, the recording unit can be adapted so that the visual dataacquisition device can be detachably attached to one or more other partsof the recording unit which are fixedly mounted on the recorder, or tothe recorder's clothing, in such manner that the visual data acquisitiondevice can be mounted at different locations on the recorder. As oneillustration of such mounting, one or more Velcro™ strips can beattached to the visual data acquisition device, and one or more matingVelcro™ strips can be attached at various locations on other parts ofthe recording unit or a single “large” mating Velcro™ strip can beprovided as part of the recording unit. For example, as discussedelsewhere herein (see FIG. 6 and the accompanying description), arecording unit according to the invention can be implemented so thatcomponents of the recording unit are positioned within a backpack thatis worn by the recorder: in such an embodiment of a recording unit, oneor both of the straps of the backpack could include a Velcro™ strip sothat the visual data acquisition device can be attached at variouslocations on the strap or straps. As can be readily appreciated, thereare a variety of other ways in which a visual data acquisition devicecan be attached to a recorder in a similar manner. For example, matingsnaps can be used in a manner analogous to the use of Velcro™ stripsdescribed above. Or, a spring-loaded clip can be attached to the visualdata acquisition device and used to clip the visual data acquisitiondevice to the recorder's clothing. There are many other possibilities.

Another way in which location-adjustable mounting of the visual dataacquisition device can be accomplished is by attaching the visual dataacquisition device to a wearable apparatus that can be worn at differentlocations and/or with different orientations on the recorder's body sothat the visual data acquisition device is mounted at differentlocations with respect to the recorder's body. As one illustration ofsuch mounting, the visual data acquisition device can be attached toheadgear (e.g., helmet, headband or head harness) that is worn by therecorder (see FIG. 4). The recorder can change the location at which thevisual data acquisition device is mounted on the recorder by changingthe orientation of the headgear on the recorder's head. Similarly, thevisual data acquisition device can be attached to an armband or legband(or other, similar apparatus that can be mounted on the recorder's armor leg) that can be worn at different locations, or with differentorientations, on the recorder's arm or leg.

Two or more of the above-described mechanisms for attaching the visualdata acquisition device to a recorder can be combined. Such combinationcan advantageously provide particularly flexible mounting of the visualdata acquisition device. For example, mating Velcro™ strips (or othersuitable means of detachable attachment) can be used to attach thevisual data acquisition device at various locations on headgear, anarmband, a legband or other wearable apparatus.

As discussed above, the visual data acquisition device is preferablysmall. The visual data acquisition device can typically be made smallenough so that mounting of the visual data acquisition device on therecorder as described above can be accomplished while providing adequatestability of the visual data acquisition device.

In addition to implementing the recording unit so that the visual dataacquisition device can be mounted at different locations on therecorder, it can be desirable to mount the visual data acquisitiondevice on the recorder so that the orientation of the visual dataacquisition device (and, thus, the filming direction) is adjustable(“orientation-adjustable mounting”). Most generally, the visual dataacquisition device can be mounted so that the visual data acquisitiondevice can be rotated about each of three orthogonal axes (e.g., pan,tilt and rotational axes), i.e., the mounting mechanism for the visualdata acquisition device provides three rotational degrees of freedom.However, increasing the number of rotational degrees of freedom maynecessitate a more complex and/or expensive mounting mechanism. Thus,the number of rotational degrees of freedom provided may involve atrade-off between the additional flexibility provided and the increasedcomplexity and/or cost.

Orientation-adjustable mounting can be combined with location-adjustablemounting to provide extremely flexible mounting of the visual dataacquisition device. In particular, the ability to mount the visual dataacquisition device so as to have adjustable orientation can provide evengreater flexibility in choosing the location on the recorder at which tomount the visual data acquisition device, since the orientation of thevisual data acquisition device can be adjusted, for any mountinglocation, to produce a variety of recording points of view.

The commonly owned, co-pending U.S. patent application Ser. No.09/225,993, entitled “Adjustable Mounting Mechanism for a DirectionalDevice with Particular Application to Mounting a Camera on a Person,” byJ. Daniel Psomas and Kimberly H. Johnson, filed on Jan. 5, 1999,describes mechanisms for location-adjustable and orientation-adjustablemounting of a visual data acquisition device on a recorder that can beused with a recording unit according to the invention. The disclosure ofthat application is incorporated by reference herein.

When the visual data acquisition device is mounted so that theorientation of the visual data acquisition device can be adjusted, itcan be useful to enable the recorder to readily identify the orientationof the visual data acquisition device at any point in time. This can beaccomplished, for example, by forming a mark on the visual dataacquisition device or a housing in which the visual data acquisitiondevice is positioned. The mark can correspond to, for example, the topof the visual frame of a lens of the visual data acquisition device. Itcan be desirable to form the mark near the periphery of the visual dataacquisition device or housing because such positioning can facilitateviewing of the mark by the recorder and/or because the increaseddistance from the rotational axis of the visual data acquisition deviceenables precise control of the rotational orientation of the visual dataacquisition device to be accomplished more easily. It can also bedesirable to form the mark on a surface of the visual data acquisitiondevice or housing that substantially faces the recorder's head, so thatviewing of the mark by the recorder is facilitated. A mechanism foridentifying the orientation of the visual data acquisition device can beparticularly useful when the visual data acquisition device or housingis generally symmetric in appearance, since such appearance tends tootherwise conceal the rotational orientation of the visual dataacquisition device.

In keeping with the general goal of the invention that the visual dataacquisition device require little attention from the recorder, it may beadvantageous for the visual data acquisition device to be implemented sothat various aspects of the device control occur automatically. Forexample, it can be desirable to use a visual data acquisition device inwhich the focusing mechanism, white balance, aperture and/or gain of thevisual data acquisition device are automatically adjusted, as requiredby the recording conditions, to increase the likelihood of obtaining ahigh quality visual recording. Such automatic compensation can enablethe recorder to, for example, move from a relatively dark area (e.g., aroom in a house) to an area with relatively more light (e.g., a yardoutside the house) without having to make adjustments to the visual dataacquisition device to account for the change in ambient lightingconditions. Additionally, some digital visual data acquisition devicescan enable such compensation (e.g., adjustment for regions of light andshadow) to be made within an image being recorded, further enhancing thequality of the visual recording. However, providing such automaticcontrol may necessitate use of an undesirably expensive and/or largevisual data acquisition device (e.g., providing an automatic focusingmechanism necessitates a relatively large and expensive lens). Thus,automatic control of visual data acquisition device operating parametersmay be desirable for some embodiments of a recording unit according tothe invention and not for others.

It may also be desirable for the visual data acquisition device to havezoom capability, thus providing a recorder with additional flexibilityin framing the visual content to be recorded. However, provision of zoomcapability necessitates use of a larger lens than would otherwise benecessary, which may add undesirable size and weight to the recordingunit. Further, since use of zoom capability requires attention from therecorder, the provision of such capability in the visual dataacquisition device may be deemed undesirable as being contrary to thegoal of the invention to minimize recorder interaction with therecording unit. If zoom capability is provided, then, consistent withthe foregoing goal, it can be desirable to establish a default lensangle that is believed to do the best overall job of capturing desirablecontent in the visual recording, so that the recorder will relativelyinfrequently feel the need to use the zoom control to expand or reducethe lens angle. For example, it is believed that in many applications itis desirable to establish a default lens angle that is the widestpossible lens angle that can be used with the visual data acquisitiondevice without producing unacceptable distortion of the recorded visualimage. (What constitutes unacceptable distortion can be influenced bythe availability and cost of post-event processing of the visualrecording data, as discussed further below, to reduce or eliminatedistortion introduced by the use of a wide angle lens.)

Generally, any desired magnitude of lens angle can be used in a visualdata acquisition device according to the invention up to, and including,a 360° angle lens. Increasing the lens angle mitigates the detrimentaleffects resultant from movement (shakiness) of the camera duringrecording. Increasing the lens angle also enables more visual content tobe captured. (Aside from conveying a greater amount of information, thecapture of a larger amount of visual content can also be useful incoordinating visual recordings obtained simultaneously by differentrecorders, since the possibility of overlapping visual content in thedifferent recordings is increased.) Notwithstanding the foregoing, itmay be desirable to limit the lens angle. As the lens angle increases,the distortion at the edges of the recorded image typically increases.(For digital video cameras with on-board signal processing capability,e.g., compression, such distortion may be non-existent or relativelyminor for even very wide lens angles; such cameras may obtain visualrecording data of acceptable quality even for a lens angle of 360°.)Though there are techniques that can be used to correct such distortion(discussed in more detail below with respect to the description ofpost-event processing techniques), as the amount of the distortionincreases, the correction becomes computationally more expensive.Further, as the lens angle is increased, the appearance of motion by thevisual data acquisition device (i.e., the recorder) is distorted so thatsuch motion appears to be faster than it would appear to be.Additionally, as the lens angle is increased, the level of detail in thevisual recording data decreases. Increasing the lens angle can alsodistort the appearance of translational motion in the visual recording,making such motion appear faster than it actually is. Further, as thelens angle is increased, the possible mounting locations of the visualdata acquisition device may become more limited; for example, using a360° angle lens may require mounting the visual data acquisition deviceon top of the recorder's head to enable the full field of view of thevisual data acquisition device to be unobstructed. For a particularrecording unit, the magnitude of the lens angle can be chosen in view ofthe intended use(s) of the recording unit and the above-describedtrade-off between advantages and disadvantages of increasing the lensangle. Illustratively, for many uses of a recording unit according tothe invention, a lens angle no wider than about 120° is desirable fortypical current analog video cameras.

Illustratively, a visual data acquisition device for use in a recordingunit according to the invention can be embodied by a Super CircuitsPC-17YC video camera. The Super Circuits PC-17YC is an analog videocamera that includes various automatic camera control features, such asautomatic gain control, automatic white balance control and automaticiris control that can compensate for changes in the ambient visualrecording conditions. The Super Circuits PC-17YC video camera includes apickup unit (including lens and other optical apparatus) interconnectedwith the electronics (one or more printed circuit boards) that are usedto perform the compensation functions. The pickup unit can be separatedfrom, and movably attached to (as described above and in theabove-referenced U.S. patent application Ser. No. 09/225,993), theprinted circuit boards on which the camera, electronics are formed sothat the recording orientation of the pickup unit can be adjusted asdesired. The Super Circuits PC-17YC video camera can be mounted on therecorder as described above. The printed circuit boards can be coveredwith an appropriate cover to protect the circuitry from contaminants andto mask their presence. A connector is provided that electricallyconnects the video camera electronics (as well as an LED, discussedbelow, for indicating whether the camera is being supplied with power)to a power supply and, in the case of the video camera electronics, to adigital disk recorder (discussed in more detail below with respect tothe description of data processing devices used in a recording unitaccording to the invention) that processes (e.g., converts from analogto digital) the visual recording data obtained by the video camera.

The Chugai CEC-100 video camera is another example of a video camerathat can be used to embody a visual data acquisition device in arecording according to the invention. The Chugai CEC-100 video camera isconstructed and operates in a manner similar to that described above forthe Super Circuits PC-17YC video camera.

As indicated above, the visual data acquisition device can be a digitaldevice instead of an analog device. For example, digital video camerassuch as are commonly used in videoconferencing systems can be used toembody a visual data acquisition device in a recording unit according tothe invention. Additionally, CMOS digital video cameras that arebeginning to be commercially available can advantageously be used with arecording unit according to the invention. The use of a CMOS digitalvideo camera can in particular be advantageous—particularly asdevelopments in technology decrease the chip size necessary to implementspecified functionality—because of their relatively small size and lightweight.

A recording unit according to the invention can include more than onevisual data acquisition device (e.g., multiple video cameras). Asdescribed further below, to accommodate multiple visual data acquisitiondevices, it may be necessary to add one or more components to therecording unit and/or to modify an existing component to enableprocessing of the additional visual recording data.

When multiple visual data acquisition devices are used, in general it isdesirable to coordinate the mounting positions of the devices to obtaina desired combination of viewpoints. For example, it may be desirable tomount the visual data acquisition devices so that significantlydifferent points of view are obtained so that as much different contentcan be recorded as possible. To accomplish such a goal, it may beadvantageous to mount the visual data acquisition devices on differentparts of the recorder's body.

The use of multiple visual data acquisition devices mounted on differentparts of a recorder's body can be implemented to produce recording unitshaving particular types of recording capability. For example, arecording unit could be implemented so that a first visual dataacquisition device is mounted on a part of the recorder's body thatgenerally moves very little, such as the chest or shoulder, while asecond visual data acquisition device is mounted on the front of therecorder's head. The first visual data acquisition device can obtain arelatively stable recording of the event (because of the relative lackof movement of the body part on which the first visual data acquisitiondevice is mounted), albeit one which may not always include the contentof most interest to the recorder. The second visual data acquisitiondevice can obtain a recording of the events that will usually includemore content of interest to the recorder than the recording obtained bythe first visual data acquisition device (because the point of view ofthe recording constantly tracks the recorder's viewing direction), butthat may not always be as stable as desired (due to, for example, quickturns of the recorder's head).

Or, for example, a recording unit could be implemented so that a firstvisual data acquisition device is mounted on a part of the recorder'sbody that generally moves very little, such as the chest or shoulder,while a second visual data acquisition device is mounted on therecorder's wrist or hand. As above, the first visual data acquisitiondevice can obtain a relatively stable recording of the event, though therecording may not always include the content of most interest to therecorder. The second visual data acquisition device, mounted on the handor wrist of the recorder, can be easily directed by the recorder torecord content of interest during the event. Though the use of thesecond visual data acquisition device requires more interaction by therecorder to obtain useful content (if the recorder does not make aconscious effort to move his hand or wrist to position the second visualdata acquisition device in the proper direction, the recorded contentwill usually not be of interest and/or will be framed poorly), it isexpected that such content will often be of greater interest than thecontent obtained by the first visual data acquisition device. Further,the recorder need make such effort to position the second visual dataacquisition device only when desired (which may, of course, be not atall). Thus, the presence of the second visual data acquisition deviceaffords the recorder the option to obtain a recording including contentof somewhat greater interest at the expense of devoting more attentionto the operation of the recording unit.

A recording unit including multiple visual data acquisition devices canalso be implemented so that at least one of the visual data acquisitiondevices (a “secondary” visual data acquisition device which is notintended to be the visual data acquisition device obtaining the primaryvisual recording) is attached to the recorder so that the visual dataacquisition device can easily be either detached or moved to enable highattention use by the recorder. For example, a visual data acquisitiondevice can be detachably attached (using mechanisms as describedelsewhere herein) to a belt or strap of the recording unit. Or, a visualdata acquisition device can be attached with a cord (wire) to the restof the recording unit so that the visual data acquisition device can bekept in a pocket of the recorder's clothing when not in use. In eithercase, when the recorder sees something of particular interest, therecorder can grab the secondary visual data acquisition device and beginusing the secondary visual data acquisition device in a directed way toobtain a desired visual recording.

It may be desirable to include features in a secondary visual dataacquisition device that are not deemed desirable or appropriate in theprimary visual data acquisition device. For example, it may be desirableto include zoom capability in a secondary visual data acquisitiondevice, but not in a primary visual data acquisition device, since useof the former is assumed to be a high attention use consonant with theuse of a zoom capability, while the latter is assumed to be a lowattention use in accordance with the primary focus of the invention.

b. Audio Data Acquisition

Generally, any suitable audio data acquisition device (e.g., microphone)can be used in a recording unit according to the invention. Eitheranalog or digital audio data acquisition devices can be used.

In keeping with the goal of minimizing the need for interaction by therecorder with the recording unit, the audio data acquisition device,like the visual data acquisition device, is desirably adapted to bemounted on (e.g., attached on or worn by, as opposed to held) therecorder. Since the audio data acquisition device can usually beimplemented by a very lightweight device (typically, one that is lighterthan the visual data acquisition device), and since movement of theaudio data acquisition device has a relatively small effect on thequality of the audio recording obtained, the stability of the mountingof the audio data acquisition device and the stability of the location(e.g., body part) at which the audio data acquisition device is mountedare usually not a concern.

It is believed to be preferable (though not necessary) to establish thecharacteristics of the audio data acquisition device, in conjunctionwith the location and orientation of the mounting of the audio dataacquisition device, so that primarily sounds from in front of therecorder are recorded. It is believed preferable to bias the audiorecording in favor of recording sound from in front of the recorder forseveral reasons: i) a person or persons with whom a recorder is having aconversation will typically be in front of the recorder; ii) a recorderwill typically face the parties participating in a conversation thatdoes not include, but is of interest to, the recorder; iii) a recorderwill often face the source of other sounds in which the recorder has aninterest; and iv) conversations that occur behind a recorder aretypically either conversations which the recorder does not want to hearor are conversations which the parties to the conversation may notintend the recorder to hear.

In view of the above-described preference, it is desirable that theaudio data acquisition device or devices of a recording unit accordingto the invention be mounted somewhere on the front of the recorder. Ifmore than one audio data acquisition device is used, it is desirablethat the audio data acquisition devices be mounted approximatelysymmetrically with respect to the center of the recorder and that theaudio data acquisition devices be mounted at some distance from thesides of the recorder. Mounting the audio data acquisition device ordevices in this way results in the body of the recorder shielding thedevice(s) to some extent from sound emanating from behind the recorder,thus biasing the content of the audio recording in favor of soundproduced in front of the recorder. It may also be desirable to mount theaudio data acquisition device(s) with respect to a visual dataacquisition device of the recording unit so that the directionality ofthe audio data acquisition device(s) changes in correspondence withchanges in the filming direction of the visual data acquisition device.

The height at which an audio data acquisition device is mounted can alsoaffect the quality of the audio recording data obtained. For example, itcan be desirable to mount an audio data acquisition device on a recorderat about chest level, shoulder level or some height in between. Thisheight is desirable because it is near the height at which speechemanates from the recorder (and from some other speakers). Mounting ator just above chest level also enables the audio data acquisition deviceto pick up spatial imaging of the audio that is similar to that which isexperienced by the recorder's ears (e.g., sound wave reflections fromthe recorder's chest).

Additionally, it can be desirable to implement the audio dataacquisition device using a directional microphone that is located andoriented on the recorder so that one of the opposing directions in whichthe microphone has greatest sensitivity extends approximately straightahead of the recorder. The use of a directional microphone may reduceto, some extent the importance of the particular positioning of theaudio data acquisition device as described above (though it is stilldesirable to use the recorder's body to shield the microphone fromsounds emanating from directly behind the recorder, i.e., in the otherdirection of greatest sensitivity of the microphone). The use of adirectional microphone as described above has been found to enhance theability to record the voices of other people, particularly when thosepeople are relatively distant from the recorder.

An omni-directional microphone can also be used in a recording unitaccording to the invention. The use of an omni-directional microphoneincreases the importance of using the body to shield the microphone fromsounds that are expected to be unwanted or relatively unimportant.Nevertheless, it can be desirable to use an omni-directional microphonefor several reasons. First, directional microphones are typically long(which may make them undesirably obtrusive, both to the recorder and toothers near the recorder) and achieve directionality through use ofcomplex sound cancellation mechanisms that minimize or eliminate soundfrom the sides of the microphone. As discussed further below, forseveral reasons it is desirable to use a lavalier microphone in arecording unit according to the invention; however, since a lavaliermicrophone is very small, it is difficult to implement a lavaliermicrophone that is highly directional. Second, an omni-directionalmicrophone acquires audio recording data that better represents thespatiality of the actual sounds being recorded than does a directionalmicrophone.

It can be advantageous to implement the recording unit so that the audiodata acquisition device can be mounted at different locations on therecording unit. This may be desirable, for example, to accommodatedifferent heights and body shapes of recorders while positioning theaudio data acquisition device at a desired location in view ofconsiderations such as those given above. Such adjustable mounting ofthe audio data acquisition device can be accomplished in any suitablemanner, such as, for example, detachably attaching the audio dataacquisition device to other parts of the recording unit which arefixedly mounted on the recorder, or attaching the audio data acquisitiondevice to an object which can be adjustably mounted on the recorder.

As one example of adjustable mounting of an audio data acquisitiondevice, the audio data acquisition device can be mounted on a flexibleclip that can be attached to, for example, the recorder's clothing or toanother part of the recording unit (e.g., a strap). The flexible clipand strap can be constructed, as can be appreciated by those skilled inthe art, so that, absent application of an external force, the clip isheld in place on the strap by frictional force between the clip andstrap. The clip and strap can also be constructed so that, uponapplication of a force of appropriate magnitude, the clip can slidealong the strap to enable the position of the clip (and thus the audiodata acquisition device) to be adjusted as desired.

As another example of adjustable mounting of an audio data acquisitiondevice, Velcro™ strips can be used to attach the audio data acquisitiondevice to a part of the recording unit. Since the audio data acquisitiondevice is typically rather small and may not have adequate surface areato which to attach a Velcro™ strip, it can be easier to simply enclose awire connecting the audio data acquisition device to the remainder ofthe recording unit between two mating Velcro™ strips that are attachedto parts of the recording unit. Similarly, the wire can be attached toan appropriate surface on the recorder or a part of the recording unitusing tape, instead of Velcro™ strips. Since the audio data acquisitiondevice is typically lightweight as well, such attachment is usuallyenough to stably secure the audio data acquisition device to therecording unit, though it is desirable that the section of attached wirebe relatively close to the audio data acquisition device so that theaudio data acquisition device does not dangle too freely. Theabove-described mounting using Velcro™ strips or tape can be implementedon the backpack straps or the headgear of the particular embodiments ofa recording unit described elsewhere herein.

An audio data acquisition device can be adjustably mounted in any of avariety of other ways. Moreover, in general, adjustable mounting of anaudio or visual data acquisition device can be accomplished using any ofthe ways described herein for either of such devices.

It is also desirable that the characteristics of the audio dataacquisition device, in conjunction with the location and orientation ofthe mounting of the audio data acquisition device, be established sothat the audio recording accurately reproduces the spatial distributionof sound that is perceived by the recorder (given the desired biastoward recording sound from in front of the recorder). To this end, itcan be desirable to use more than one audio data acquisition device.Generally, any number of audio data acquisition devices can be used in arecording unit according, to the invention. With appropriate placementon the recorder, an increase in the number of audio data acquisitiondevices improves the spatiality of the audio recording. However, as thenumber of audio data acquisition devices increases, the cost ofproviding those devices increases, the complexity of integrating theaudio data acquisition devices into the operation and construction ofthe recording unit increases, the audio data acquisition devices aremore likely to distract the recorder or interfere with the recorder'smotion, and the audio data acquisition devices tend to become a moreostentatious part of the recording unit (contrary to the goal of makingthe recording unit inconspicuous). The particular number of audio dataacquisition devices used is chosen in view of the tradeoff between thesedisadvantages and the enhanced audio recording quality.

In general, it is desirable that multiple audio data acquisition devicesbe mounted approximately symmetrically with respect to the center of therecorder. Further, the accuracy of the reproduction of the spatialdistribution of sound is improved as the distance between the audio dataacquisition devices is increased. In particular, sound as heard by therecorder can be accurately reproduced by positioning audio dataacquisition devices symmetrically with respect to the center of therecorder with a separation approximating the separation of therecorder's ears. However, such positioning causes the audio dataacquisition devices to be positioned relatively closely to the sides ofthe recorder, thus reducing the above-described desired shielding effectproduced by the body of the recorder. The particular distance betweenaudio data acquisition devices can be chosen in view of this trade-offbetween minimizing the recording of sound from behind the recorder andmaximizing the fidelity of the spatial distribution of sound in therecording.

Some microphones are constructed so that a single microphone can enablean audio recording to be obtained that reproduces to some degree thespatial distribution of sound that is perceived by the recorder. Such amicrophone can be used with a recording unit according to the invention.However, such a microphone generally needs to be positioned at somedistance from the body of the recorder in order to work effectively.This can be undesirable because, for example, such positioninginterferes more with the recorder's participation in the event, is morenoticeable to others, and is more susceptible to bumping that canproduce extraneous noise and misorient the microphone. Additionally, themounting of such a microphone is more complicated to implement.

The audio data acquisition device can be chosen so that a desiredbandwidth of audio signals can be recorded. It has been found to bedesirable to use an audio data acquisition device that records audiosignals over the entire frequency spectrum that is audible to the humanear an audio recording so obtained is believed to provide an auralexperience that is more pleasant than that obtained using an audio dataacquisition device that records audio signals over a more limitedbandwidth range. However, the audio recording obtained by such an audiodata acquisition device may be more sensitive to static, buzzes, humsand other background noise. If such noise is deemed undesirable, anaudio data acquisition device that records a more limited bandwidth ofaudio signals can be used. Alternatively, such noise can be removed orreduced by appropriate processing, such as filtering or equalization,either during recording or as part of the post-event processing ofrecording data.

For many applications (outdoor applications, in particular) it is highlydesirable that the audio data acquisition device be adapted so as tominimize wind noise and other ambient noise. For microphones, this canbe done by covering the microphone with a conventional windscreen. Itmay be necessary to modify a conventional windscreen to ensure that thewindscreen remains in position on the microphone during the expectedconditions of use of the recording unit. (Such modification can beaccomplished by those skilled in the art.)

Illustratively, an audio data acquisition device for use in a recordingunit according to the invention can be embodied by one or more lavalierstereo omni-directional microphones (e.g., a Lectret 5200). Lavalieromni-directional microphones are desirable for use with a recording unitaccording to the invention because they are relatively small andlightweight, they are easy to mount on a recorder, they need notprotrude from the recorder when mounted, and they have been found toconsistently produce a high quality audio recording (in particular, theLectret 5200 microphone is characterized by a flat, extended frequencyresponse that is suitable for reproduction of voice or musicalinstruments). An example of a stereo directional microphone that can beused in a recording unit according to the invention is the Sanken CSS-5.

c. Other Data Acquisition

A recording unit according to the invention can also include dataacquisition devices for acquiring other types of data. For example, arecording unit according to the invention can include one or morephysiological monitoring devices that obtain data regarding a physicalcharacteristic of a recorder. Generally, in view of the desirablecharacteristics of a recording unit according to the invention (e.g.,unobtrusive, small, lightweight, wearable, easily-operable, relativelylow power consumption), any physiological monitoring device can be usedwith a recording unit according to the invention.

For example, a physiological monitoring device can be used to monitor,preferably on an ongoing basis, physical characteristics of a recorder,such as, for example, heart rate, galvanic skin-response, EEG, ECG, bodytemperature, respiration rate and/or volume, blood oxygen level and/orlevel of CO₂ in exhaled breath, that can be used to infer informationregarding the recorder's physical and/or mental state during recording.Such information can be useful in a: variety of ways. For example,monitoring one or more of the, above-mentioned physical characteristicscan enable the recorder's emotional state to be inferred (e.g., when therecorder is excited or bored). This information can then be used toenable identification of parts of a recording that have a relativelyhigh probability of being interesting or uninteresting. This can beparticularly useful when a large amount of recording data has beenobtained, since there may only be a relatively small amount of therecording data that is of real interest. In particular, physiologicaldata can be advantageously used with retrospective and predictive marks(discussed elsewhere herein) to facilitate determination of whetherrecording data associated with a retrospective or predictive mark infact represents content of interest (e.g., recording data temporallyproximate to the recording data designated by the mark can be searchedto identify recording data obtained while the recorder was experiencinga particular emotional state, such as excitement or fear). (Positiondata can also be used for this purpose.) Additionally, suchidentification of the nature of content represented by particularrecording data can be used itself to mark parts of the recording, e.g.,mark parts of the recording that are expected to be interesting. It mayalso be desirable to simply display such physiological data, or displayan indicator of the recorder's emotional state that corresponds to thephysiological data n excitement is sensed, display an indication ofexcitement and, perhaps, the level of excitement).

A physiological monitoring device can also be used to monitor otherphysical characteristics for other purposes. For example, it may bedesirable to measure forces exerted on, or by, a recorder at one or morepoints on the recorder's body. This data could be displayed to therecorder during recording, perhaps to facilitate the recorder inanticipating fatigue or monitoring over-exertion of a weak or injuredbody part (e.g., an injured knee), or to allow the recorder to gauge thelevel of exercise that the recording is receiving during the event (thismay be useful, for example, when the recording unit is being used duringphysically rigorous activities, such as riding a bicycle or hiking). AnEMG device, which measures neural activity to muscles, can also be usedfor these purposes.

Multiple physiological monitoring devices can be included as part of arecording unit according to the invention. A recording unit can includemore than one of a particular type of physiological monitoring device(e.g., multiple EMG devices) and/or can include multiple different typesof physiological monitoring devices (e.g., devices for measuring heartrate, galvanic skin response and/or body temperature). Measurements frommultiple physiological monitoring devices can be compared, combined orotherwise analyzed together to enable or facilitate inference ofinformation regarding the recorder based on the physiological,measurements. Measurement(s) from one physiological monitoring devicecan also be compared to measurement(s) from a second physiologicalmonitoring device to increase or decrease the level of confidence ininformation inferred about the recorder based on the measurement(s)obtained by the first physiological monitoring device. For example,measurement of an elevated heart rate of the recorder may indicate thatthe recorder is excited or has just engaged in physical exertion. Ameasurement of galvanic skin response can assist in determining which ofthese two possible explanations (or another explanation) is the mostlikely explanation for the recorder's elevated heart rate. Respiratorymeasurements can enable further refinement of the inference regardingthe reason for the recorder's physical state.

It may also be desirable to use a biometric device as part of arecording unit for identification purposes, e.g., to render a recordingunit non-operational unless an approved person is identified by thebiometric device. Biometric identification can also be used to controlaccess to recording data and/or augmenting data. Biometric devices thatcan be used for this purpose include, for example, fingerprint scanningdevices, retinal scanning devices or faceprint scanning devices.

Those skilled in the art will recognize that there are other ways inwhich the physiological monitoring and/or biometric data described abovemight be used with a recording unit according to the invention, eitherduring recording or during post-event processing and/or display of arecording. Further, those skilled in the art will appreciate that thereare other types of physiological monitoring and/or biometric data thatcan be obtained and productively used by a recording unit according tothe invention.

A physiological monitoring or biometric device for use in a recordingunit according to the invention can be implemented so that the recordermust initiate operation of the device (i.e., acquisition ofphysiological or biometric data). Alternatively, the physiologicalmonitoring or biometric device can be implemented so that the deviceoperates automatically, acquiring data whenever the recording unit ispowered on or recording (and, if applicable to the particular device,e.g., a biometric device used for identification, when the devicedetects the presence of an input to the device's sensor(s)). The lattertype of physiological monitoring or biometric device is more in keepingwith the goal of making a recording unit according to the invention alow attention device. However, the former may be desirable to give therecorder more control over the acquisition of physiological or biometricdata, which recorders may generally deem to be more intrusive and, insome situations, undesirable.

4. Data Processing and Storage

A wide variety of data processing and storage devices can be used in arecording unit according to the invention. The data processing andstorage devices used must be controllable by the system controller,which typically is embodied by a computer processing device. Theparticular devices used can depend upon the intended applications forthe recording unit, as well as the types and characteristics of theother devices that are part of the recording unit.

For example, it may be necessary to provide a device that performs A/Dconversion of recording data if, for example, the recording unitincludes one or more analog data acquisition devices (e.g., analog videocamera, analog microphone(s)) and a digital data storage device (e.g.,hard disk). Such processing can be accomplished using conventionaltechniques and apparatus.

Since the amount of visual recording data obtained is typically verylarge, it can be desirable to provide a device that performs compressionof the visual recording data before the visual recording data is storedon the data storage device, thus reducing the amount of data that mustbe stored to represent a visual recording. (While it is also possible tocompress the audio recording data, compression of the audio recordingdata is typically less useful than compression of the visual recordingdata, since the amount of visual recording data obtained per unit timeis typically much larger than the amount of audio recording dataobtained per unit time.) Data compression is particularly advantageousin a small portable device, such as a recording unit according to theinvention, since the data storage device(s) included therein arerelatively small and, therefore, have a relatively limited storagecapacity. A recording unit according to the invention can be implementedso that the amount of compression is predetermined as part of theoperating software of the recording unit or, as discussed further below,so that the amount of compression can be specified by the recorder. Inparticular, it can be desirable to implement the recording unit so thatthe visual recording data is always compressed a specified minimumamount, since such compression increases the amount of recording datathat can be stored by the recording unit and, therefore, the duration oftime that the recording unit can be used before the provision foradditional data storage must be made (either by transferring therecording data from the recording unit to another data storage medium orby replacing the non-volatile data storage device of the recordingunit).

A variety of methods for compressing visual data are known and can beused to compress the visual recording data obtained by a recording unitaccording to the invention. Compression methods that facilitatesearching and/or playback of stored, compressed visual data arepreferred. In one embodiment of the invention, a motion JPEG (M-JPEG)compression method is used. (The Recon-11 DDR that can be used in arecording unit according to the invention, as discussed below, usesM-JPEG for compression of the visual recording data.) M-JPEG compressesframes of visual data individually, thus enabling images to be readilylocated and reconstructed from stored, compressed visual recording data.Additionally, for the processing power that is typically available in arecording unit according to the invention, M-JPEG produces a goodcombination of image quality and compression ratio. However, othercompression methods, such as MPEG or wavelets, can also be used.

Though compression of the visual recording data is desirable-because itreduces the amount of data that must be stored to represent a visualrecording, compression of the visual recording data usually results inloss of some of the image quality of the visual recording. Thus, it canbe desirable to selectively compress the visual recording data, varyingthe amount of compression of the visual recording data for differentparts of the visual recording. In particular, it can be desirable toenable the recorder to control the amount of compression of the visualrecording data as the recording is being obtained. The recorder can thenchoose to obtain a relatively high quality visual recording (i.e.,compress the visual recording data little or not at all) of things thatare of particular interest or that the recorder believes may benefitmore from an improvement in visual recording quality, and a lowerquality visual recording (i.e., compress the visual recording data arelatively large amount) of other things. Or, if the recorderanticipates recording for a relatively long period, of time, therecorder may choose to compress all of the visual recording data (and,perhaps, compress as much as possible), so that the amount of time thatthe recording unit can be used is increased. The recording unit can beimplemented so that the recorder can choose one of several discretecompression ratios, can choose any compression ratio along a continuumbetween a maximum and minimum compression ratio, or can choose a desiredimage quality which corresponds, in turn, to a particular compressionratio or sequence of compression ratios (i.e., the amount of compressionto produce images of a desired quality can vary depending upon thecontent of the images).

The difference in duration of visual recording that can be stored by arecording unit can be drastic, depending upon the amount of visualcompression used. Illustratively, when the above-described SuperCircuits PC-17YC video camera is used for visual recording and a harddisk having a 9 gigabyte storage capacity is used for non-volatilestorage of recording data, approximately 20 minutes of visual recordingcan be stored on the hard disk when the visual recording data iscompressed a minimal amount (e.g., a compression ratio of about 4:1). Ifthe visual recording data is compressed as much as possible while stillretaining a recognizable visual recording (e.g., a compression ratio ofabout 54:1), approximately 4.5 hours of visual recording can be stored.It is desirable to compress the visual recording data so that therecording unit can be used for the entirety of most events for which useof the recording unit is intended (e.g., so that 2-3 hours of visualrecording data can be stored by the recording unit). Thus, it maybe-desirable to restrict the recorder to a minimum compression ratio(e.g., a compression ratio of about 24:1) that ensures that at least aminimum duration (e.g., 2 hours) of visual recording can be stored bythe recording unit, and a maximum compression ratio that ensures thatthe visual recording will have at least a minimum level of imagequality.

To conserve power in the recording unit, it can be desirable to controla non-volatile data storage device of the recording unit so that thenon-volatile data storage device is not fully operational (e.g., thedisk or disks of a hard disk are being rotated) when recording data hasnot been acquired and stored, or stored recording data has not beenaccessed for playback, for a predefined period of time. The non-volatiledata storage device can be controlled in such manner in response toappropriate control instructions from the system controller or otherprocessing device.

Depending on the particular data acquisition and data processing devicesused, it may also be necessary to use a pre-amplifier to amplify thesignal representing the data obtained by the data-acquisition deviceprior to inputting the signal to the data processing device or devicesthat perform some or all of the above-described functions (e.g., A/Dconversion, compression). For example, when the above-described lavaliermicrophone is used as an audio data acquisition device together in arecording unit with the Recon-11 DDR described below, it is necessary touse a pre-amplifier to amplify the audio signal from microphone level toline level prior to inputting the signal to the Recon-11 DDR.

A digital disk recorder (DDR) can be used to accomplish both A/Dconversion and digital data compression in a recording unit according tothe invention. Illustratively, a DDR that can be used with the inventionis a Recon-11 made by Fast Forward Video, located in Irvine, Calif. ThisDDR is a relatively low power, small, portable device, thus making itparticularly suitable for use in a recording unit according to theinvention. Additionally, this DDR can perform A/D conversion of bothaudio and visual data. This DDR can also be programmed (as can readilybe done by those skilled in the art) to enable the DDR to control anon-volatile data storage device to which the DDR is connected, inresponse to instructions from a system controller, so that the datastorage device is operated at a low power condition at times whenrecording data is not being, or will not imminently be (in accordancewith the characteristics of the data storage device), obtained. This DDRcan also be programmed (again, as can readily be done by those skilledin the art) to enable the DDR to record other types of data (e.g., GPSdata, compass data, time codes) with the visual recording data.

Typically, the data processing device or devices of a recording unitaccording to the invention are adapted to process recording data from adefined set of data acquisition devices of the recording unit. Forexample, small DDR's are typically adapted to process recording datafrom a single visual data acquisition device and several (e.g., 2-4)audio data acquisition devices. It may be desirable, however, for thedata processing device(s) to be capable of processing recording datafrom one or more additional data acquisition devices. For example, itmay be desirable for a recording unit including a small, inexpensive DDRas above to be able to receive recording data from an additional visualdata acquisition device for storage by the recording unit. This may benecessary, for example, to enable the recording unit to include anadditional visual data acquisition device (as discussed above). Or, itmay be desirable to enable processing of recording data received by therecording unit from another recording unit to facilitate, for example,display and/or storage of a recording being obtained by such otherrecording unit. It may also be desirable for a recording unit includinga small, inexpensive DDR as above to be able to process data acquired bya visual data acquisition device for storage by the recording unit,while simultaneously outputting data previously stored by the recordingunit for display by the recording unit.

One way of enabling processing of recording data from one or moreadditional devices is to provide redundant data processing device(s)(e.g., multiple DDR's) and integrate the operation of those dataprocessing devices with each other. The integration of multiple DDR'sinto a recording unit according to the invention, for example, can bereadily accomplished by those skilled in the art in view of thedescription herein. However, a DDR (e.g., a Recon-11 DDR as describedillustratively above) can be a significant source of power consumptionin a recording unit according to the invention; the use of multiple suchDDR's necessitates the provision of a larger power supply (thus makingthe recording unit larger and heavier) in order to enable the recordingunit to be used for the same amount of time without recharging orreplacing the power supply, or acceptance that the recording unit can beused for a significantly shorter period of time without recharging orreplacement of the power supply. Further, the addition of dataprocessing devices to the recording unit undesirably increases theweight and bulk of the recording unit.

Instead of adding additional data processing devices, the existing dataprocessing devices can be modified (by modifying the data processingdevice hardware and/or the software used to control operation of thedata processing devices) to enable processing of recording data acquiredby, or to be provided to, additional devices. This can be desirable toeliminate the extra power consumption, weight and bulk associated withthe addition of additional data processing device(s). However, modifyingthe data processing devices to accommodate additional devices can be amore complex (and therefore, perhaps, undesirable) task than adding andintegrating additional data processing devices.

Further, while the current state of DDR technology is such thatprocessing by a single, small DDR of recording data acquired from, or tobe provided to, additional devices (especially visual data acquisitiondevices) as described above mandates use of a DDR that is undesirablyexpensive, heavy and bulky for use in a recording unit according to theinvention, further development of DDRs can be expected to result in theproduction of a sufficiently inexpensive, small and lightweight DDR thatcan process the extra recording data from the additional devices.

Another possibility for enabling processing of recording data fromadditional data acquisition devices is to modify the data processingdevice(s) so that only some of the data from each data acquisitiondevice is processed. For example, NTSC video, commonly acquired at 30frames per second, is acquired by interlacing the acquisition of pairsof fields (each field comprised of half the data of a frame of videodata) at 60 fields per second. A data processing device adapted-toenable data acquisition from an NTSC video data acquisition device canbe modified to acquire data from two NTSC video data acquisition devicesby causing the data processing device to process only one of each pairof fields acquired by each video data acquisition device. While thisreduces the resolution of the images acquired by each video dataacquisition device, that may be deemed an acceptable price to pay toenable acquisition of video from an additional data acquisition device.The above-described technique can readily be used for analog recording.The above-described technique can also generally be used for digitalrecording, though use of an MPEG compression algorithm may make thetechnique very difficult or impractical to implement.

If the data processing device(s) of the recording unit are adapted toenable processing of multiple recordings (in particular, multiple visualrecordings), it can be possible to further adapt the recording unit toenable the recording data of the multiple recordings to be stored. Thiscan be done by appropriately adapting the data processing device(s)(e.g., the computer programs that effect control of the operation of thedata processing device(s)), as can be accomplished by those skilled inthe art, to cause the recording data to be stored.

It may be desirable to identify acquired audio recording data thatrepresents sounds that are anticipated to be of little or no interest orimportance to the recorder, so that such audio recording data can bediscarded and not stored on a data storage device of the recording unit,thereby freeing up storage capacity for obtaining other recording data(e.g., audio recording data from additional audio data acquisitiondevices). For example, as discussed above, it is anticipated that itwill typically be desirable to acquire audio recording data primarilyrepresenting sounds that emanate from a location in front of therecorder. However, if a recording unit according to the inventionincludes omni-directional microphone(s), the acquired audio recordingdata will include a substantial amount of audio data representing soundsemanating from locations other than those in front of the recorder. Evena directional microphone positioned to primarily sense sounds thatemanate from in front of the recorder will sense sound from the sidesand behind the recorder. It may be desirable to eliminate such audiorecording data. It may also be desirable to eliminate other audiorecording data that is anticipated to be of little or no interest orimportance to the recorder, e.g., audio recording data representingsounds emanating from particular sources, such as particular speakers.

If a recording unit according to the invention includes multiplemicrophones, a beam forming method can be used to discriminate thesounds of interest or importance from other sounds. A beam formingmethod can be performed by the system controller or data processingdevice(s) operating in accordance with a computer program stored in thedata storage device. (It may, alternatively, be desirable to provide aseparate processing device or devices, e.g., one or more digital signalprocessors, to perform beam forming.) Beam forming can, for example,enable isolation of the location of the source of an audio signal fromthe difference in time of receipt of the audio signal at eachmicrophone. Once the location of the source of the audio signal has beenisolated, a decision can be made, based upon one or more predeterminedcriteria, as to whether to discard or store the audio data representingthe audio signal. The implementation of beam forming is known to thoseskilled in the art and is discussed in more detail below with respect tothe description of post-event processing of recording data.

If beam forming is used in a recording unit according to the invention,it may be necessary to use more robust and/or complex processingdevice(s) and/or methods than would otherwise be used, or add additionalprocessing device(s) to the recording unit, to perform the beam formingcomputations. If additional audio data acquisition devices are added tothe recording as a consequence of performing beam forming, theimplementation of beam forming in a recording unit according to theinvention may also necessitate more complex data storage device(s)and/or methods. Those skilled in the art of implementing a beam formingmethod will understand how to specify the data processing and storagecapabilities needed to perform the beam forming computations.

The quality of the discrimination of a beam forming method is improvedas more microphones are used to provide audio data for use in themethod. However, each additional microphone adds to the required dataprocessing and storage capabilities. Moreover, the incrementalimprovement in beam forming results decreases with each additionalmicrophone. Further, the addition of a microphone will increase theamount of recording data to be stored. The number of microphones usedcan be determined in view of these considerations.

It may be desirable for several reasons to implement a recording unitaccording to the invention so that beam forming is performed. Forexample, as indicated above, beam forming can reduce the data storagecapacity required for the audio recording data obtained by a particularaudio data acquisition device or devices. Thus, the use of beam formingin a recording unit according to the invention can enable additionalaudio data acquisition devices to be used in the recording unit withlittle or no attendant increase in data storage capacity requirements.The acquisition of audio recording data by one or more additional audiodata acquisition devices is desirable because the likelihood ofobtaining audio content of interest is increased and/or a higher qualityaudio recording display can be produced from the acquired audiorecording data.

Additionally, as suggested above, beam forming can be particularlyuseful if the microphones of a recording unit according to the inventionare omni-directional, since such microphones can be expected to acquirerelatively more audio data representing sounds that are not of interestor importance to the recorder. In fact, the use of beam forming canenable relatively cheaper (as compared to directional microphones)omni-directional microphones to be used while still obtaining forstorage on the data storage device a set of audio recording data thatis, in terms of the spatial distribution of the audio recording,comparable to (or even, perhaps, better than) a set of audio recordingdata that could be obtained using directional microphones.

However, the implementation of beam forming in a recording unitaccording to the invention can have some disadvantages. For example, theuse of beam forming to discard some audio recording data prior tostorage creates the risk that desirable audio recording data may bediscarded. (As indicated above, beam forming can be used to eliminateaudio data that is anticipated—not known—to be of little interest.)Additionally, the use of beam forming to discard some audio recordingdata may impair the spatial fidelity of the audio recording. Further,the addition of processing capability to the recording unit to enableperformance of the beam forming calculations adds to the cost of therecording unit. Thus, it may be preferred to implement a recording unitaccording to the invention without the capability to perform suchprocessing of the audio recording data.

It may also be desirable to process physiological monitoring dataobtained by a recording unit according to the invention. For example,physiological monitoring data can be lanalyzed to determined theemotional state of the recorder, e.g., the recorder's level ofexcitement, whether the recorder was startled, etc. Such derivativeinformation can be displayed on a recording display device of therecording unit during recording. Additionally, it may be desirable tostore this derivative information on a data storage device of therecording unit. In particular, it may be desirable to store thisderivative information in lieu of the raw physiological data in order tominimize the data storage capacity required for storing physiologicalmonitoring information. Alternatively, or in addition, it may bedesirable to store the raw physiological data, so that the rawphysiological data is available for post-event analysis.

A “data storage device” of a recording unit according to the inventioncan be embodied by one or more devices. Any appropriate devices,including both digital and analog devices, can be used. A recording unitaccording to the invention must include one or more non-volatile datastorage devices. Below, the characteristics of non-volatile data storagedevice(s) for use in a recording unit according to the invention, aswell as some exemplary embodiments of such non-volatile data storagedevice(s) are discussed in more detail. A recording unit according tothe invention can also include one or more volatile data storagedevices. For example, a random access memory (RAM) may be used to bufferdata during recording.

The non-volatile data storage device(s) of a recording unit according tothe invention are used to store instructions and/or data that arenecessary to accomplish the functions of the recording unit or that areacquired during operation of the recording unit. This includes, ofcourse, the recording data. The non-volatile data storage device(s) canalso be used to store data (augmenting data) other than the recordingdata (e.g., data identifying, or corresponding to, the recording data),such as, for example, data identifying the time at which the recordingdata was obtained, data identifying the recording unit (and/or recorder)that obtained the recording data, data representing the position and/orvelocity of the recording unit at the time the recording data wasobtained, data representing a mark to be associated with the recordingdata, and/or physiological monitoring and/or biometric data representingphysical characteristics of the recorder. The non-volatile data storagedevice(s) are also used to store instructions and/or data (e.g.,computer programs) used to effect operation of components—in particular,the system controller—of the recording unit.

Recording data and augmenting data can be stored together on a singlenon-volatile data storage device. Recording data and augmenting data canalso be stored on different non-volatile data storage devices. In thelatter case, it can be desirable to establish a temporal correspondencebetween the augmenting data and the recording data in order to minimizeor eliminate the need for post-event processing to temporallysynchronize the augmenting data and recording data. Such a temporalcorrespondence can be established using methods as described below forestablishing a temporal correspondence between the auxiliary data andthe recording data obtained by a hybrid recording unit according to theinvention.

In general, it is desirable that a recording unit according to theinvention have a large data storage capacity. In particular, this isdesirably so for the data storage device(s) used to store the recordingdata, so that the recording unit can record events of relatively longduration. Data storage capacity can be increased by using a data storagedevice with larger data storage capacity or by using multiple datastorage devices. However, typically, increasing data storage capacity(in either of the foregoing ways) results in an increase in the size andweight of the data storage device(s). Additionally, increasing datastorage capacity by adding one or more data storage devices necessitatesa more complex integration of the data storage device(s) with the restof the recording unit (e.g., system controller). As indicated above, itis generally desirable that the recording unit be as small as possible;in keeping with this goal, it is desirable that the data storagedevice(s) be as small as possible. The data storage capacity of the datastorage device(s) of a recording unit according to the invention can beestablished in view of the above considerations, together withconsideration of the intended use(s) of the recording unit and thecharacteristics (e.g., size and weight) of the other components of therecording unit. It is anticipated that, for many applications of theinvention, it is desirable that a non-volatile data storage device of arecording unit according to the invention include sufficient datastorage capacity to allow, in view of the particular implementation andoperation of other components of the recording unit, several hours(e.g., 2-5 hours) of recording data to be stored.

Since the recording unit is a portable device and therefore more likelythan a stationary device to be subject to jarring movements (as a resultof, for example, sudden movement by the recorder, jarring of therecorder by impact with another person or an object, dropping of therecording unit, or bumping of the recording unit into another object),it is also desirable that the data storage device(s) be implemented sothat the data storage device(s) are relatively insensitive to shocksassociated with such impacts. For example, impact-resistant hard disksthat are used in portable computers can advantageously be used in arecording unit according to the invention.

As indicated above, any of a variety of appropriate devices can be usedto embody a “data storage device” of a recording unit according to theinvention. For example, the non-volatile data storage device can be aconventional hard disk. Illustratively, in one embodiment of a recordingunit according to the invention, a Seagate Barracuda hard disk having a6 or 9 gigabyte capacity can be used to embody a non-volatile datastorage device of the recording unit according to the invention. Aparticular Barracuda hard disk that is compatible with the Recon-11 DDRand has a 9 gigabyte data storage capacity is the hard disk having partno. ST 39173-N. The casing of this half-height hard disk isapproximately 1 inch high, 4 inches wide and 5.75 inches deep, making ita relatively small device that is suitable for use with a recording unitaccording to the invention.

In general, both IDE and SCSI hard disks can be used in a recording unitaccording to the invention. When the above-described Recon-11 DDR isused in a recording unit according to the invention, and the hard diskinterfaces directly therewith, a SCSI hard disk must be used. In otherembodiments of the invention, however, it can be desirable to, use anIDE hard disk. For example, there are a variety of high-capacity,shock-resistant IDE hard disks that have been developed for use inportable devices. Illustratively, IBM has developed small (i.e., using2.5 inch disks), lightweight, low power, IDE-compatible hard disksreferred to as Travelstar hard disks. In particular, the IBM Travelstarhard disk designated as the 14GS has a 14 gigabyte data storage capacityand is rated to withstand an operating shock of about 125 g's and anon-operating shock of about 400 g's; such a hard disk can beadvantageously used in a recording unit according to the invention.

Non-volatile data storage devices other than a hard disk can also beused. For example, the visual recording data can be recorded onto avideotape. Such an implementation has an advantage that videotape is notsusceptible to the data destruction that can occur with a hard disk ifthe disk surface of the hard disk is damaged as a result of jarring ofthe hard disk. On the other hand, recording data stored on videotapecannot be searched and played back as easily as can data stored on ahard disk. The use of a videotape may also entail some other limitationson the processing of recording data by the recording unit: for example,the use of videotape prevents user-selected variable compression. Eitheranalog videotape (such as a videotape using the Hi-8 format) or digitalvideotape (such as a videotape using the mini-DV, DV-CAM, DVC-PRO, DVCPRO 50, digital beta cam or D-VHS format) can be used.

A recording unit according to the invention can also be implemented touse a removable data storage medium, such as a recordable once DVD(DVD-R) disk or a recordable/writable DVD (R-W DVD) disk, as anon-volatile data storage device. Such an implementation has anadvantage that the recording unit could be modified to enable the DVDdisk to be easily removed and replaced with a new DVD disk. This wouldenable the recording unit to be used “in the field” for a longer periodof time, limited only by the number of DVD disks that it is feasible tocarry.

Solid-state memory, such as flash memory, can also be used as anon-volatile data storage device in a recording unit according to theinvention. Flash memory can be desirable because, unlike a hard disk,there are no moving parts that can fail. Additionally, flash memoryconsumes relatively little power. Flash memory is also relatively smalland lightweight. In view of the foregoing, it can be desirable to useflash memory in a recording unit according to the invention,particularly as the data storage capacity of flash memory increases withfurther developments in that technology.

The above discussion illustrates some of the possibilities for storageof data in a recording unit according to the invention. Data storagetechnology continues to evolve, existing technologies being improved(e.g., data storage capacity increased, size reduced, cost reduced) andnew technologies being developed. It can readily be appreciated thatsuch developments will produce new, currently non-existent devices thatcan advantageously be used in a recording unit according to theinvention.

5. Communication with Other Recording Units and Recorders

a. Communication Between Recording Units

Each recording unit can include a transmitter that enables signalsrepresenting the recording data (and, if desired, other related data)obtained by that recording unit to be broadcast from the recording unit.Similarly, each recording unit can include a receiver that enablesreceipt of signals broadcast from other recording units that representthe recording data (and, if desired, other related data) obtained bythose recording units. The transmitters and receivers enable recordingdata to be communicated between recording units during recording of anevent by the recording units, thus enabling a recording unit to displayand/or store, during the event, a recording being obtained by anotherrecording unit. The recording data signals transmitted by thetransmitter of a recording unit can also be received by the receiver ofthat recording unit and used to generate a display of the recordingbeing obtained by the recording unit. (This can be advantageous, forreasons explained further below.) While a recording unit may includeonly a transmitter or a receiver—thus limiting that recording unit toeither transmitting its own recording data or receiving recording datasignals from other recording units—it is anticipated that it willusually be desirable that all recording units include a transmitter andreceiver.

Since, typically, a multiplicity of recording units will be transmittingrecording data signals at the same time, the transmitter and receiver ofa recording unit according to the invention are preferably adapted toenable transmission and reception of recording data signals over amultiplicity of transmission channels, so that each recording unit cantransmit recording data signals over a channel that is not being used byanother recording unit. In general, any number of channels can be used.The number of channels can be established in view of expectationsregarding the number of recording units that will typically be usedtogether to record an event. Additionally, the number of channels may belimited by the available communications bandwidth (particularly sincebandwidth consumptive visual data is being transmitted). The number ofchannels can be established in view of considerations such as thosegiven above, as well as any other considerations relevant to aparticular implementation of a recording unit according to the inventionor the intended use of the recording unit. Illustratively, fourtransmission channels can be used.

Additionally, to ensure that only signals representing recording datafrom a recording unit according to the invention can be received by areceiver of a recording unit according to the invention, the recordingdata can be encoded prior to transmission in a manner that identifiesthe transmitted signal as one transmitted by a “compatible” recordingdevice (i.e., a recording unit according to the invention). Further, therecording data can be more specifically encoded so that the recordingdata is identified as coming from a particular recording unit or groupof recording units. (It is anticipated that the latter, more specific,encoding will typically be done, since it is anticipated that recorderswill only want to share recordings with other recorders in their group.)Correspondingly, the receiver can be adapted so that only signals thathave been properly encoded (indicating that the signals have beentransmitted by a recording unit according to the invention) can bereceived. Such encoding can be accomplished in any of a variety ofwell-known ways. For example, the signals can be encrypted (using anyappropriate encryption algorithm), transmitted surrounding a specifiedcarrier frequency, transmitted in unique data packets or transmittedusing a spread-spectrum technique. Moreover, if the transmitted dataincludes data other than recording data (such as GPS position data,physiological monitoring data and/or biometric data, as discussedelsewhere herein), the transmitted signals will inherently be closely(perhaps uniquely) identified with recording units according to theinvention.

As indicated above, the recording data signals received by a recordingunit from another recording unit can be used to generate a display ofthe recording obtained by that other recording unit and/or used to storerecording data obtained by the other recording unit in the data storagedevice of the receiving recording unit. However, it may desirable toallow storage in a recording unit only of recording data representingthe recording(s) being obtained by that recording unit. This may bedesirable to minimize the possibility of failing to store recording datarepresenting interesting content. For example, as recording occurs, twoor more recorders (and, in a worst case, all recorders) may decide thatthe recording being obtained by a particular recording unit is of mostinterest, and may, as a result, control their recording unit to causethat recording data to be stored. Unexpectedly, interesting activityoccurs for which recording data is acquired by a recording unit or unitsthat is storing data representing the recording(s) being obtained byanother recording unit. Since recording data representing theinteresting activity is not being stored, the recording of theinteresting activity is lost. Thus, by allowing each recording unit tostore data representing the recording(s) obtained by another recordingunits, one of the advantages of having multiple recording unitsrecording an event from multiple perspectives can be diminished oreliminated. By allowing each recording unit only to store data obtainedby that recording unit, the chances of recording interesting activityare increased. Though such a limitation does preclude a recorder fromstoring recording data obtained by other recording units that may bemore interesting than the recording data obtained by the recorder'srecording unit, it is anticipated that this limitation will often be oflittle or no detriment, since it will often be the case that therecorders have a sufficiently close relationship that the recorders can,and will, exchange recording data after the event is over, thus enablingeach recorder to obtain, if desired, part or all of a recording obtainedby another recording unit. It is to be understood, however, that arecording unit according to the invention can also be implemented sothat the recording unit can store recording data obtained by anotherrecording unit, and, further, so that the recorder can, on an ongoingbasis, choose from which recording unit to store recording data.Further, with appropriate implementation of the data recordingdevice(s), a recording unit according to the invention may beimplemented so that recording data from multiple recording units can berecorded simultaneously; this capability may more likely be included inrecording units according to the invention as recording devices (e.g.,DDR's) evolve to make such capability more technologically simple (andinexpensive). However, to prevent the storage by others of a recordingof something sensitive or personal, it may be desirable to accompany theinclusion of the capability of storing recording data from anotherrecording unit with an additional capability for a recorder to designaterecording data acquired by that recorder's recording unit as data thatcannot be recorded (even, perhaps, by that recorder's recording unit).It may further be desirable to implement this privacy capability so thatthe recorder can prevent the transmission of such recording data in thefirst place. Additionally, the recording unit is preferably enabled toallow a recorder to simply turn off the transmitter at any time.

Transmitters and receivers for use in recording units according to theinvention can be embodied by conventional television signal transmittersand receivers. In general, either analog or digitaltransmitters/receivers can be used. While analog devices are moreinexpensive, digital devices are more reliable (e.g., digital devicestypically suffer less from problems with interference between channels,digital communication protocols can include redundancy codes, it iseasier to insert additional data into the digital data packet stream).The use of digital transmitters and receivers also makes it easier tobroadcast other data (e.g., marks) with the recording data.Additionally, it may be desirable to match the type oftransmitters/receivers with the type of data acquisition devices beingused, so that it is not necessary to perform an A/D or D/A conversionprior to transmission of the data.

Illustratively, 2.4 gigahertz (the “nominal” frequency) analogtelevision signal transmitters and receivers can be used, suchcomponents being readily commercially available. Generally, however,transmitters and receivers operating at other parts of the frequencyspectrum can be used. Each of the transmission channels operates at afrequency near the nominal frequency.

Recording data can be transmitted as acquired. Alternatively, it can bedesirable to process the recording data prior to transmission to otherrecording units. For example, if the data acquisition devices being usedin the communicating recording units are analog and thetransmitters/receivers are digital, or vice versa, it is necessary toperform an A/D or D/A conversion, as appropriate, prior to transmittingthe data. Additionally, the recording data can be compressed beforebeing transmitted. (The recording data can be compressed by an amountfor transmission from the recording unit that is different than theamount by which the recording data is compressed for storage by therecording unit.) Compression of the recording data prior to transmissionhas the advantage of reducing the amount of data transmitted (which may,for example, enable more channels of recording data to be transmitted,i.e., more recording units can simultaneously transmit data). However,compression can degrade the quality of the images generated from therecording data. Whether or not to compress the recording data—and, ifso, how much—can be decided in view of this tradeoff.

Data other than recording data can also be exchanged between recordingunits. For example, it may be desirable to coordinate the recordingsobtained by recording units so that frames of visual recording dataobtained by different recording units are temporally synchronized (e.g.,so that a frame begins at the same time for each recording unit). Thiscan be done by transmitting a synchronization pulse from a “master”recording unit to each of the other recording units (“slave” recordingunits). (Synchronizing analog visual data acquisition devices in thisway is referred to as “genlocking.”)

b. Communication Between Recorders

As discussed above, it is envisioned that a recording unit according tothe invention will frequently be used to record a group event that isalso being recorded by other people (recorders) who are using arecording unit according to the invention. A recording unit according tothe invention can include an audio communication device that enablesspoken communications between and among recorders during the event. Theinclusion of an audio communication device can be particularly desirableas a mechanism to enable communication when circumstances make directspoken communication impractical, impossible or undesirable. While anaudio communication device is generally desirable to enablecommunication of any type between recorders, an audio communicationdevice may be particularly desirable to facilitate coordination ofrecording by different recorders to, for example, increase thelikelihood of obtaining content of interest, ensure that all members ofthe group are being recorded during the event, minimize redundancy inthe content of different recordings, or acquire content that can be usedto create interesting visual effects when displayed at a later time.While such coordination can occur to a certain extent as a result ofsharing recordings, as described above, the use of an audiocommunication device makes such coordination easier.

Like other components of a recording unit according to the invention, itis desirable that the audio communication device require littleattention from the recorder (e.g., be easy to operate). The audiocommunication device can be made wearable as a means to achieve thisgoal. Additionally, again like other components of a recording unitaccording to the invention, it is desirable that the audio communicationdevice be small and lightweight. It is also desirable that the audiocommunication device have a communication range that is adequate toensure acceptable communication over the maximum anticipated distancebetween recorders during the event (it is anticipated that adequaterange will not be a problem for most audio communication devices thatmay be used in a recording unit according to the invention).

Additionally, it can be desirable that the audio communication device beunobtrusive. Obtrusiveness of the audio communication device includesseveral aspects: characteristics (e.g., loudness) and control of theaudio output; physical interference with, or distractiveness to, therecorder; and distractiveness to people in the vicinity of the recorder.For example, it can be desirable that the audio communication device beembodied so that audio communication received from another recorder isnot provided to the recorder loudly enough to distract the recorder orpeople in the vicinity of the recorder from participation in the event,or to be undesirably prominent in the audio recording obtained by arecording unit. The size, weight and mounting location (if wearable) ofthe audio communication device can also desirably be controlled so thatthe presence of the audio communication device is not overly noticeableto the recorder. It may also be desirable to implement the audiocommunication device (together with the control interface device,discussed below), so that the recorder can control whether audiocommunications can be received by the audio communication device. Forexample, a recording unit can be implemented so that a signal (e.g., anaudible tone and/or a display of the identity of a person attempting tocommunicate with the recorder) is provided to the recorder when anotherrecorder has indicated a desire to transmit an audio communication, acontrol interface device of the recording unit including a capabilitythat allows the recorder to either receive or not receive the audiotransmission, such decision being communicated to the recorder desiringto transmit an audio communication.

On the other hand, making the audio communication device unobtrusive canhave disadvantages. If a person interacting with a recorder is unawarethat the recorder is receiving audio communications from otherrecorders, the person may be put off when the recorder reacts to acommunication from another recorder in a manner that seems, absentknowledge of the communication, out of place or rude. Or, if a person isaware that the recorder is receiving audio communications from otherrecorders, but can't hear the content of those communications, they mayfeel uncomfortable because of a perception of exclusion or ofsecretiveness on the part of the recorder. Such considerations areparticularly important for an apparatus, such as a recording unitaccording to the invention, that is intended to be used in socialsituations. The particular implementation of an audio communicationdevice according to the invention can be chosen in view of theabove-described advantages and disadvantages associated with making theaudio communications device obtrusive or unobtrusive.

An audio communications device of a recording unit according to theinvention can be embodied any of a variety of known devices. Theparticular device used and the manner of integrating the device with therecorder and the rest of the recording unit can be established in viewof the considerations discussed above and below.

For example, a headset including speakers and a microphone, such as wornby telephone operators and receptionists, can be used. The use of such aheadset can be desirable because the headset is designed to be awearable device and is relatively lightweight. The headset alsominimizes or eliminates the distractiveness of audio communications toothers in the vicinity of the recorder. Additionally, the headsetrequires no action on the part of the recorder other than speaking andlistening (which, of course, is a necessary minimum involvement of therecorder) and is, in that way, unobtrusive. However, the physicalpresence of the speaker/microphone headset may be distracting to therecorder or to other people interacting with the recorder.

A two-way radio or cell phone can also be used. These devices can bemade small and light enough that the device can be carried easily in apocket of the recorder's clothing, or can be worn by the recorder (e.g.,attached in a manner similar to that described above with respect to thevisual and audio data acquisition devices, such as by clipping on ashoulder strap). If carried in a pocket, the device is not distractingto others, but may require more interaction by the recorder to enableuse of the audio communication device than is desired. Illustratively, aNextel phone having a walkie-talkie mode that makes use of a cellularnetwork can also be used (as long as the recording units are withinrange of a Nextel cell site). The use of a Nextel phone can beparticularly advantageous because of the extensive communication rangeafforded (much greater than that of a typical two-way radio). Ofparticular utility when more than two recorders are using a recordingunit according to the invention, a Nextel phone can be operated in amode that enables communication from a recorder to be broadcast to allrecording units in a defined group.

A speaker/microphone lapel unit (such as used by police officers) orother similar unit that can be worn near the head can also be used. Suchspeaker/microphone units are wearable, small, lightweight and relativelyunobtrusive. The speaker/microphone unit would desirably be mounted inthe vicinity of the recorder's mouth (e.g., a lapel unit could bemounted on the backpack straps of the particular embodiment of arecording unit described below with respect to FIG. 6) and would includea pushbutton for activating the microphone. Device(s) for transmittingand receiving communication are interconnected with thespeaker/microphone unit and can be embodied, for example, in anapparatus that can be carried around in a pocket of the recorder'sclothing.

Transmitters and receivers that can be used to communicate recordingdata between recording units can also be used to enable communicationbetween recorders. A recording unit (the “transmitting recording unit”)can be adapted to enable the transmitter of the recording unit tocommunicate a signal or signals representing communicative information(as described further below) that can be received by the receiver of oneor more other recording units (the “receiving recording unit(s)”). Forexample, a transmitting recording unit can be adapted to transmitcommunicative information that causes emission of an audible tone,display of a visual cue (e.g., lighting of an LED indicator) and/orproduction of a haptic sensation (e.g., vibration, using mechanisms suchas have been used in cell phones or pagers) by a receiving recordingunit, such tone and/or visual cue having a predetermined meaning. Thepredetermined meaning can be, for example, that a recorder (a “receivingrecorder”) using a receiving recording unit should listen to therecording being transmitted from the transmitting recording unit. Therecorder using the transmitting recording unit (the “transmittingrecorder”) can then talk, such speech being heard as part of therecording transmitted from the transmitting recording unit to thereceiving recording unit(s), thereby enabling the transmitting recorderto communicate to the receiving recorder(s). Such a tone and/or visualcue can also have other meanings, such as, for example, that thereceiving recorder should watch the visual recording being transmittedby the transmitting recording unit, that the receiving recorder shouldlook at the transmitting recorder, or that the receiving recorder shouldlook or listen in some other manner (e.g., “look around”). Suchcommunication between recorders can be useful, in particular, when it isdifficult or impossible for one or more receiving recorders to otherwisehear or see the transmitting recorder. Further, recording unitsaccording to the invention can be implemented so that differentcommunicative information can be transmitted that produces correspondingdifferent tones and/or visual cues. This may be useful, for example, toenable identification of the transmitting recorder when (as is expectedwill typically be the case) there are multiple possible transmittingrecorders. This may also be useful, for example, to enableidentification of which of multiple types of communicative informationhas been received by the receiving recording unit. Communicativeinformation can also be transmitted by implementing a control interfacedevice of a recording unit so that the recorder can specify desiredtextual information to be transmitted to one or more other recordingunits, which, upon receipt of that information by receiving recordingunit(s), is displayed by a display device (which can be an audio displaydevice, a visual display device, or both) of the receiving recordingunit(s). For example, the textual information can be displayed as a textoverlay on a visual display device or as digitized or synthesized speechby an audio display device.

It can also be desirable to provide a capability for a recorder tocommunicate with only a specific one of the other recording units. Thiscan be implemented in a variety of ways. For example, a controlinterface device of a recording unit can include a special button thatis used to indicate to a specified other recording unit (by, forexample, causing a transmitter to transmit a signal to that recordingunit that causes emission of an audible tone by that recording unit)that a message is being communicated to that recording unit. The controlinterface device of the first recording unit can then be used asdescribed above to specify communicative information that is receivedand displayed by a display device of the second recording unit. Anotherway to provide such one-on-one communication is to designate aparticular channel of the audio communications devices used forcommunication among all of the recorders as a channel for use in suchone-on-one communication. While this alternative is less private thanthe previously described implementation, it may be easier and lessexpensive to implement, since the necessary devices are already part ofthe recording units.

6. Recording Display

A recording unit according to the invention can include one or moredevices that display a recording to the recorder as the recording isbeing obtained. While a recording display device is not a necessary partof a recording unit according to the invention, a recording displaydevice (and, in particular, a visual recording display device) is highlydesirable, since the recording display device can be used to monitor thecontent of a recording as the recording is being obtained, therebyenabling a recorder to control the recording unit as necessary to obtaindesired content and to verify that the recording unit is operatingproperly (both of which are particularly useful when the recorder firstbegins recording or when activity of particular interest is occurring).Additionally, when a recording unit is implemented so as to be able toacquire recording data being obtained by other recording units, thecapability of displaying those recordings is important to fully realizethe benefits of acquiring that recording data. Further, the ability toview recordings obtained by other recorders can be used, together withthe audio communication capability described above, to enable a group ofrecorders to coordinate their recording.

A display device or devices can be provided that display a visualrecording, an audio recording, or both. Generally, any appropriatevisual and audio display devices can be used to implement a visualrecording display device and audio recording display device in arecording unit according to the invention.

In general, a displayed recording can be a recording obtained by therecording unit of which the display device(s) used to display therecording are part, or a recording obtained by another recording unit.The recording data used by a recording unit to generate a display of arecording obtained by another recording unit is obtained via receptionby the recording unit's receiver of appropriate recording data signalsfrom such recording unit, as described above. The recording data used bya recording unit to generate a display of the recording being obtainedby that recording unit can also be obtained by the recording unit'sreceiver. When the recording data for all displayed recordings isobtained by the recording unit's receiver, control of which recording isdisplayed can be effected by selecting an appropriate channel forreception of the recording data signals for the desired recording.

Alternatively, recording data used to generate a display of therecording being obtained by the recording unit's own data acquisitiondevices can be obtained from those data acquisition devices (perhapsafter processing by a data processing device of the recording unit).This approach necessitates the inclusion of a switching device in therecording unit to enable switching between recording data obtained bythe receiver and recording data obtained by the data acquisitiondevice(s). While this approach will typically enable a higher qualitydisplay to be generated of the recording being obtained by the recordingunit's own recording devices (because there are no transmission lossesor signal interference as occur with the transmission of the recordingdata signal from the recording unit's transmitter to the recordingunit's recorder), it will also increase the cost and complexity of therecording unit as a result of the need to provide the switchingcapability. If digital data acquisition and transmission devices areused, rather than analog, the degradation of the transmitted signal maybe relatively minimal, so that the quality of the display generated fromthe recording data signal received by the receiver that represents therecording unit's own recording data is of the same or nearly the samequality as the display generated from recording data that is obtainedfrom the recording unit's data acquisition devices.

A switching device as described above may also be necessary when therecording unit has been adapted to enable multiple recordings to beobtained (e.g., recordings from multiple visual data acquisitiondevices).

A recording unit according to the invention can also be implemented sothat recording data that has been previously obtained and stored by therecording unit can be retrieved and used to generate a display of apreviously acquired part of the recording. The system controller, uponreceipt of an appropriate instruction from the recorder via the controlinterface device, causes the recording data to be retrieved from thedata storage device and transmitted to a recording display device fordisplay.

a. Visual Recording Display

In view of the fact that the recording unit is a portable apparatus, andin keeping with the goal of the invention that the recording unit beunobtrusive, the visual recording display device is preferably embodiedby a relatively small and lightweight visual display device. Forexample, in some embodiments of the invention, a recording unit isimplemented with the intent that the visual recording display device becarried in a pocket of the recorder's clothing when not in use; in suchembodiments, the visual recording display device must be sufficientlysmall to enable this to be done, typically resulting in a visualrecording display device that consumes relatively little power. In someembodiments of the invention, as described further below, a recordingunit is implemented so that a visual recording display device is mountedon the recorder; in such embodiments, the visual recording displaydevice should be sufficiently small and lightweight that the mountedvisual recording display device does not unduly encumber or distract therecorder. Since power consumption decreases with display size, using asmall visual recording display device also has the virtue of reducingpower consumption. (Particularly in embodiments of the invention inwhich the visual recording display device can be carried in therecorder's pocket, the power consumption of the visual recording displaydevice can be greatly reduced by simply turning off the visual recordingdisplay device when in the recorder's pocket.)

As other components of a recording unit according to the invention, thevisual recording display device can be adapted to be mounted on (e.g.,attached on or worn by, as opposed to held by) the recorder. Forexample, the visual recording display device can be mounted on the backof the hand, on the wrist, or on the forearm of the recorder. This couldbe accomplished, for example, by attaching (e.g., strapping) the visualrecording display device to a glove or a wristband. Illustratively, thevisual recording display can be mounted on a wristband having opposingends on which Velcro™ is formed, the opposing ends of the wrist bandbeing secured to each other when the wristband is wrapped around arecorder's wrist. Such a wristband can be advantageous (particularlyvis-a-vis a glove-mounted visual recording display device) because itcan readily be used by recorders having different wrist sizes.

Another way in which the visual recording display device can be attachedto the recorder is by attaching the visual recording display device toone end of a cord, the other end of which is attached to the recorder,e.g., to a waist belt or strap that is part of the recording unitaccording to the invention. To prevent the visual recording displaydevice from dangling awkwardly, mating Velcro™ strips (one attached tothe visual recording display device, the other attached to the recorder)or other appropriate attachment mechanism can be used to attach thevisual recording display device to the recorder when not in use.

It may also be desirable to mount the visual recording display device inthe vicinity of the eyes of the recorder (e.g., on the head, neck orshoulders) in a manner that enables the recorder to readily move thevisual recording display into and out of the recorder's field of view.Though mounting a visual recording display device in this manner may bemore obtrusive than mounting the visual recording display device onanother part of the body, such mounting may be desirable—as compared to,for example, mounting the visual recording display device on the hand orwrist, or implementing the visual recording display device as anapparatus to be carried in the recorder's pocket—since the recorder canview a display of a visual recording without occupying a hand, thusleaving the hand free, either to effect control of the recording unit orto enable the recorder to participate in the event being recorded. Forexample, the visual recording display device can be mounted (e.g., witha hinge or a sliding mechanism) on glasses worn by the recorder so thatthe visual recording display device can be moved back and forth betweenpositions in front of and away from one (preferable, so that the othereye is free for other viewing) or both eyes of the recorder.Alternatively, a display has recently been developed that can beembedded in eyeglasses (in a manner that is almost completely invisibleto others), such that the wearer of the eyeglasses can control whetherthe display is visible or not (in the latter case, the wearer of theeyeglasses can see normally through the eyeglasses).

A visual recording display device can also be mounted on the visual dataacquisition device. This can be useful to facilitate monitoring by therecorder of the visual recording being acquired as the recorder adjuststhe orientation, the position or a filming characteristic (e.g., zoomlens, brightness control) of the visual data acquisition device.

Since it is anticipated that the primary function of the visualrecording display device is to provide information to the recorderregarding the content of a visual recording that is being obtained by arecording unit, in general the visual recording display device need onlyproduce a quality of display that is adequate to convey to the recorderthe content of the displayed recording. Since the cost of a visualdisplay device may decrease as the quality of the visual displaydecreases, the use of a lower quality display can be desirable becauseof the reduced cost. However, notwithstanding the increased cost, it canbe desirable to provide a visual recording display device having ahigher quality display in order to provide a more pleasing viewingexperience to the recorder. (This may be particularly desirable if thevisual recording display device can be used to view the recordingsobtained by other recorders.)

As described above, a recording unit according to the invention can beimplemented so as to enable acquisition of recording data obtained byone or more other recording units. Typically, then, this enables arecorder to display any of multiple visual recordings. As describedelsewhere herein, a recording unit according to the invention can beimplemented so that the recorder can select one of multiple visualchannels for display of a corresponding visual recording by a visualrecording display device of the recording unit.

In a recording unit that has the capability to acquire visual recordingsfrom multiple recording units, it can be advantageous to implement thevisual recording display device using a device capable of simultaneouslydisplaying multiple visual recordings at one time. For example, thevisual recording display device could be implemented so that the displayscreen is partitioned into equally sized windows, each window displayinga specified visual recording. Or, the visual recording display devicecould be partitioned into windows in a manner analogous to that of thepicture-in-picture (PIP) display implemented in some television sets.However, though such simultaneous visual display is desirable. becauseit enables a recorder to be cognizant of the content of the visualrecordings being obtained by more than one recording unit, such adisplay has disadvantages. For example, it may be felt that theprovision of multiple visual recording displays is undesirablydistracting to the recorder. Additionally, a visual recording displaydevice having such multiple display capability is more complex andexpensive to implement than a visual recording display device capable ofdisplaying only a single visual recording. Further, since a visualrecording display device of a recording unit according to the inventionis preferably relatively small, the display screen of such device willbe small as well: the display screen may be too small to adequatelydisplay multiple visual recordings. Whether to provide simultaneousvisual display capability can be decided in view of such considerations.

A visual recording display device for use in a recording unit accordingto the invention can be embodied, for example, by a device that includesan LCD display. LCD displays using absorptive LCD technology can beused, as well as those using reflective LCD technology. The use of anLCD display can be advantageous because of the ready availability andrelatively low cost of LCD displays. Illustratively, a Casio VM-50 LCDTFT display can be used. This LCD display is relatively small andproduces a high quality image, though it requires a relatively largeamount of power. Such display (or any other type of display) is situatedwithin a housing and appropriately connected to other components of therecording unit (such as the receiver) as can be readily understood bythose skilled in the art.

b. Audio Recording Display

As with other components of a recording unit according to the invention,it is desirable that the audio recording display device be small.Implementing the audio recording display device together with anothercomponent of the recording unit can aid in achieving this goal. Forexample, the audio recording display device can be one or more speakersintegrated together with the visual recording display device in a singledevice. In the same vein, the audio recording display device can also beembodied by the audio output portion (speaker and related circuitry) ofthe same device used to implement audio communication between recorders.

It is also desirable that the audio recording display device beunobtrusive. Implementing the audio recording display device so as to bemountable on (e.g., attached on or worn by, as opposed to held) therecorder can aid in achieving this goal. For example, the audiorecording display device can be a small speaker mounted near the head(and, thus, ears) of the recorder (e.g., on a backpack strap, in aparticular embodiment of a recording unit as shown in FIG. 6). The audiorecording display device can also be a headset, such as is commonly usedwith a variety of portable audio devices. However, the use of a headsetmay preclude or greatly inhibit firsthand aural participation in theevent by the recorder, perhaps rendering its use undesirable. Tominimize such detrimental effect, a non-isolating headset, which can beturned off to minimize or eliminate aural interference, can be used.

7. Control Interface

The control interface device includes all of the mechanisms for enablingthe recorder to effect control of the operation of the recording unit,as well as the mechanisms for displaying information to the recorderregarding the operation of the recording unit. Illustratively, a controlinterface device for use in a recording unit according to the inventioncan be implemented in a manner similar to remote control devices used inother consumer electronic equipment, i.e., a housing including variousmechanisms (such as pushbuttons, switches, rotary knobs, and sliderbars) which a recorder can activate to cause electronic devices withinthe housing to provide instructions to the system controller to causeperformance of the functions provided by the control interface device.

The control interface device is connected to the system controller, theinput received by the control interface device being translated intoelectrical signals that are transmitted to the system controller whichthen transmits electrical signals to appropriate component(s) of therecording unit to effect operation of those components in accordancewith the input received by the control interface device. Theimplementation of methods and apparatus to accomplish suchinterconnection and communication can be accomplished by those skilledin the art, particularly in view of the description herein.

When both are present in a recording unit according to the invention,the control interface device can be integrated with a recording displaydevice or provided separately. Integration of the control interfacedevice and recording display device can be advantageous because itgenerally enables the control and display functionality to be packagedmore compactly and because the consolidation of interface functions canfacilitate interaction of the recorder with recording unit. For example,control interface functionality and recording display functionality canbe combined into a single hand-held device. (The device could be kept inthe recorder's pocket or the device could be attached to the recorder inone of the ways described elsewhere herein, e.g., attached with Velcro™to a belt or shoulder strap worn by the recorder.) Providing the controlinterface device and the recording display device in separate devicescan be advantageous because it can allow more flexibility inconstruction and use of the recording unit. For example, it may bedesirable to mount a visual recording display device near the recorder'shead, as described above, while mounting the control interface device onthe recorder's hand or wrist, attaching the control interface device toa waist belt or strap with a cord or strap, or sizing the controlinterface device so as to fit in a pocket of the recorder's clothing.

As with other components of the recording unit, it is desirable that thecontrol interface device (or devices) be implemented so as to beunobtrusive and, in particular, minimize the need for interaction by therecorder with the control interface device. Thus, it can be desirablethat a control interface device be adapted to be mounted on (e.g.,attached on or worn by, as opposed to held) on the recorder. Typically,the stability of the mounting of the control interface device is not asignificant concern as it relates to affecting the operation of thecontrol interface device; however, the control interface device shouldbe mounted securely enough to ensure not only that the control interfacedevice does not fall off of the recorder, but also that the recorder isnot concerned about the control interface device falling off of therecorder (again, with the purpose of freeing the recorder's attentionfrom the recording unit) or is not distracted by excessive movement ofthe control interface device. It can also be desirable to construct thecontrol interface device in a manner that makes the control interfacedevice unobtrusive (i.e. unnoticeable) to others. For example, thecontrol interface device can be constructed to look like a piece ofjewelry (e.g., a brooch) or some other item that people often wear(e.g., a watch).

Generally, the control interface device can be mounted on the recorderin ways that are the same as, or similar to, those described elsewhereherein for other components of the invention. It is desirable to mountthe control interface device at a location on the recorder that enablesthe recorder to easily interact with the control interface device, bothto effect control of the control interface device and to view anydisplay regarding the operation of the control interface device. Forexample, the control interface device can advantageously be mounted onthe back of the recorder's hand or on the recorder's wrist. Suchmounting of the control interface device would be useful, for example,for events in which the recorder is walking. Illustratively, the controlinterface device can be attached to a wristband that can be wrappedaround a recorder's wrist, opposing ends of the wrist band havingVelcro™ formed thereon to secure the opposing ends of the wrist band toeach other. Or, the control interface device can be attached to a glovethat is worn on the recorder's hand. Other types of mounting may bedesirable for other types of events.

It may also be desirable to implement the mounting of the controlinterface device so that the control interface device can be mounted atdifferent locations on the recorder, thus providing flexibility to therecorder in use of the recording unit. For example, the controlinterface device can include a clip that can be clipped on to a waistbelt or strap of the recording unit at different locations. Mountingwith Velcro™ can also be used to effect adjustable mounting in a manneras described elsewhere herein for other components of a recording unitaccording to the invention. Such mounting could further be implementedso that the control interface device can be pulled away from the belt orstrap (if, for example, the device is attached with a cord) or flippedup from a rest position when the recorder wishes to use the controlinterface device.

However, it is not necessary—and may not be desirable—to mount thecontrol interface device on the recorder. The control interface devicecan also satisfy the goal of being unobtrusive and minimizing requiredrecorder attention, for example, by being made sufficiently small to fitinto a pocket of the recorder's clothing, so that the control interfacedevice can be carried in the pocket and withdrawn only when it isdesired to effect control of the recording unit.

Additionally, for some applications, it may be desirable that thecontrol interface device be adapted for mounting on another object. Forexample, if the recording unit is to be used while the recorder isriding a bicycle, it can be advantageous for the control interfacedevice to be adapted to be mounted on the bicycle, e.g., on thehandlebars or the down tube. Such mounting can be accomplished in any ofa variety of ways, such as, for example, attachment with mating Velcro™strips or a clamp. When the control interface device (or any othercomponent of a recording unit according to the invention) is adapted formounting on another object, it can be particularly desirable for therecording unit to be implemented to enable wireless communication (asdiscussed further below) between the control interface device (or othercomponent) and the remainder of the recording unit.

As indicated above, the control interface device enables the recorder tocontrol operation of the recording unit. The control interface deviceincludes one or more mechanisms (e.g., pushbutton, switch) for enablingthe recorder to start and stop operation of the recording unit. Forexample, the control interface device includes a power switch thatcontrols whether power is supplied to other components of the recordingunit. Depending on the particular implementation of the recording unit,it can take several seconds or more for various components of therecording unit to reach an operational state (e.g., for a hard disk tospin up and/or a data processing device to boot up) after power to therecording unit is turned on. The control interface device can alsoinclude a record switch that controls whether recording takes place(once power has been turned on).

During recording, a recorder may suddenly realize that sensitive contentis about to be, or is being, recorded, and, as a result, desire toimmediately stop recording. If the power switch is used to stoprecording, it can take an undesirably long time to power up therecording unit for operation again. It can also take an undesirably longtime for the recording unit to execute steps required to shut down thecomponents of the recording unit. Additionally, the record switch may beimplemented so that an undesirably long interaction with the recordingunit is required to stop recording (e.g., the recorder may need to workthrough a set of control menus). In view of the foregoing, it can bedesirable to provide a readily accessible and operatable mechanism (a“kill switch”) that enables the recorder to immediately shut down powerto the visual data acquisition device to stop acquisition of recordingdata (and, if applicable, to the transmitter to stop transmission ofrecording data), without shutting down all power to the recording unit.Since the kill switch does not shut down power to all components of therecording unit, recording can begin again more quickly than if the powerswitch was used to stop recording.

The control interface device (or other appropriate component of therecording unit) can also include one or more mechanisms for indicatingthe status of various operational parameters of the recording unit. Forexample, the control interface device can include a mechanism forindicating whether the recording unit is recording or not. Such amechanism can be useful to indicate to the recorder on an ongoing basiswhether the recording unit is operating, so that the recorder does notinadvertently miss recording parts of an event that the recorder desiresto record or record parts of an event that the recorder does not desireto record, as may otherwise occur if the recorder forgets to turn therecording unit on or off, or if a component of the recording unit fails.Such a mechanism can also be useful to indicate to others that therecording unit is operating, so that people present in the vicinity ofthe recorder can take appropriate steps, if they desire, to avoid havingthemselves recorded. The control interface device (or other appropriatecomponent of the recording unit) can also include similar mechanisms forindicating whether the recording unit is turned on or not (i.e., whetherpower is being supplied to the recording unit components), and whetherthe recording unit is transmitting or not, so that the recorder canreadily monitor those aspects of the recording unit's operation. Each ofthese mechanisms can be implemented, for example, using a conventionalLED appropriately electrically connected to the appropriate electronicsof the recording unit, as readily understood by those skilled in theart. Existing camcorders, for example, typically include a similarmechanism which can be adapted for use with a recording unit accordingto the invention. Alternatively, a single mechanism can be used toconvey each of the above-described operational pieces of information,e.g., an LED that emits different light patterns. Or, a text display ofa control interface device, as discussed further below, can be used topresent operating status to the recorder (e.g., when the recording unitis recording, the text display can display “REC” or “RECORDING”). It mayalso be desirable to provide multiple mechanisms indicating the statusof a particular operating parameter. For example, an LED indicatingwhether the recording unit is recording could be provided at a locationthat is especially visible to the recorder and a second LED indicatingthe same thing could be provided at another location that is especiallyvisible to other people in the vicinity of the recorder. Additionally,it may be desirable to transmit data representing operating status toother recording units. Further, to enhance communication of the functionof the recording unit to other people in the vicinity of the recorder, alogo that conveys in some way the function of the recording unit (e.g.,“BodyCam”) can be formed on an exterior surface or surfaces of therecording unit. For example, this can clarify for people nearby themeaning of an illuminated LED indicating that the recording unit isrecording.

As indicated above, the data storage device can be controlled so thatthe data storage device is only fully operational when it is anticipatedthat recording data is about to be obtained. The control interfacedevice can include a mechanism that enables the recorder to effect suchcontrol of the data storage device. Similarly, if such functionality isimplemented in the recording unit, the control interface device caninclude one or more mechanisms that enable the recorder to shut downother components of the recording unit that are not being used, so thatpower consumption can be reduced.

If the recording unit is adapted to enable acquisition of recordingsfrom other recording units (or to obtain multiple recordings itself,such as multiple visual recordings), as described elsewhere herein, thecontrol interface device can include a mechanism that enables therecorder to specify which of the recordings to display with theabove-described recording display devices (i.e., the control interfacedevice includes a display channel selector). The control interfacedevice can also include a mechanism that enables the recorder to allowor prohibit transmission to other recording units of the recording beingobtained by the recording unit. Such mechanisms can be implemented usingany of a variety of well-known mechanisms (such as rotary knobs and/orpushbuttons) that are appropriately connected to the electronic devices(e.g., system controller) of the recording unit to effect such control,as known to those skilled in the art of constructing user interfacedevices. These mechanisms can be implemented so that particularrecording units are identified in any appropriate manner, such as, forexample, by color (which can correspond to the color of part or all ofthe recording unit), alphanumeric character, or name of recorder.

If the recording unit is adapted to enable recorder-controlled variablecompression of the visual recording data, as described above, thecontrol interface device includes a mechanism that enables the recorderto specify the amount of compression of the visual recording data. Sucha mechanism can be implemented using any of a variety of well-knownmechanisms (such as a slider bar, a set of pushbuttons, or a menudisplay), as known to those skilled in the art of constructing userinterface devices.

The control interface device can also advantageously include a mechanismto enable the recorder to specify that a particular part of therecording be marked in some manner, i.e., that marking data be storedthat is associated with and/or identifies particular recording data insome way. (Storage of marking data requires very little data storagecapacity and so will not noticeably affect the capacity of the recordingunit to store recording data.) Generally, each mark can be used tospecify a time or duration of time during the recording and can haveassociated therewith a predetermined meaning. Any of a variety ofmarking capabilities can be established, as described further below andas described in the commonly-owned, copending United States PatentApplication entitled “Flexible Marking of Recording Data by a RecordingUnit,” by Henry B. Strub et al., filed on the same date as the presentapplication and having Attorney Docket No. IR-022, the disclosure ofwhich is incorporated by reference herein. The marking mechanism of acontrol interface device can be implemented using any of a variety ofwell-known mechanisms (e.g., one or more pushbuttons, switches or knobs,or a display of one or more menus), as known to those skilled in the artof constructing user interface devices. The particular marking mechanismused may be determined, or militated toward, by the particular markingcapabilities provided, as can be appreciated from the furtherdescription of such capabilities below.

In a simple implementation of marking capability, the control interfacedevice enables specification of a single type of mark, the markindicating that a recorder considers the marked part of a recording toinclude content of particular interest. The marked part of the recordingcan be identified by the mark as the time during the recording at whichthe mark was specified. In other words, a recorder can mark content ofinterest at the moment the content is being recorded. The markingmechanism for such an implementation can simply be, for example, apushbutton.

A mark enables quick location of recording content in the temporalvicinity of the part of the recording to which the mark refers. Whatconstitutes “temporal vicinity” can be specified in any of a variety ofways. For example, a mark can be used to move to the exact time in arecording specified by the mark. Or, the mark can be used to move to atime in the recording that is a predetermined amount of time before orafter the time specified by the mark. Or, the mark can be used to causea segment of the recording to be displayed, the segment beginning at thetime specified by the mark or at a time that is a predetermined amountof time before or after the time specified by the mark, and ending atthe time specified by the mark or at a time that is a predeterminedamount of time before or after the time specified by the mark (e.g., thesegment begins 5 seconds before the time specified by the mark and ends5 seconds after that time). The temporal vicinity of a mark can bespecified as part of the post-event processing and display a recording.The temporal vicinity of a mark can also be defined as the meaning orpart of the meaning of a mark.

The invention can be implemented to enable a recorder to mark a part ofa recording other than the part being obtained at the time the recorderspecifies the mark. In particular, it is anticipated that it will beuseful to enable a recorder to specify a mark that is associated with apart of a recording previously obtained (i.e., to mark the recordingwith a retrospective mark). This can be particularly useful in view ofthe goal of the invention that the recording unit be implemented so asnot to distract the recorder from participating in the event, since arecorder may be so immersed in participating in the event that therecorder does not realize until some time after activity of interest hasoccurred (perhaps hours or days after) that it is desirable to mark therecording of that activity. Such marking can be implemented, forexample, by enabling the recorder to specify a particular time (thespecified time being constrained to be prior to the time at which themark is specified) or a duration of time (the time prior to the time atwhich the retrospective mark is specified by the amount of the durationof time being the time associated with the mark). The marking mechanismcan again be implemented, for example, by pushbuttons: one button can beused to indicate that a retrospective mark is being specified, whileother buttons (e.g., a numeric keypad) can be used to specify the amountof time prior to the present which the mark is to designate. Othermarking mechanisms can also be used, such as a display of one or moremenus. It may also be desirable to enable a range of prior times to bespecified (e.g., within 1 minute of the time that is 10 minutes prior tothe current time), to enable a confidence level to be specified with aprior time (e.g., a 50% certainty that the content of interest occurred20 minutes ago), or to enable both a range of prior times and aconfidence level to be specified. Further, the magnitude of the range oftimes and/or the confidence level can be made dependent (perhapsautomatically) upon the “recency” of the mark (i.e., the proximity ofthe marked time to the marking time at which the mark is specified). Forexample, as the duration of time between the marked time and the markingtime increases, the confidence level typically decreases. Additionally,as the duration of time between the marked time and the marking timeincreases, the importance of the use of other data (e.g., physiologicalmonitoring data, position data) in aiding the identification of theactual recording data that corresponds to a mark increases, and viceversa.

Similar to retrospective marking, the invention can be implemented toenable a recorder to specify predictive marks, i.e., to mark a part ofthe recording that will be obtained in the future. Like retrospectivemarking, predictive marking can be consonant with the goal of theinvention to minimize distracting the recorder from participating in theevent, since the recorder can mark anticipated activity of interestprior to the activity (at a presumably less interesting time), thusfreeing the recorder for participation in the activity without having tobe concerned about remembering to mark the recording of the activity.Predictive marking can be implemented in ways similar to retrospectivemarking, e.g., by specifying a particular time in the future (absolutetime) or by specifying an amount of time from the present (relativetime). Likewise, similar marking mechanisms can be used. And, as withretrospective marking, predictive marking can be implemented to enablespecification of, with the predictive mark, a range of future timesand/or a confidence level.

The control interface device can also be implemented so as to enablespecification of multiple types of marks, each mark having a differentmeaning. A marking mechanism can be provided for each type of mark(e.g., multiple pushbuttons, one corresponding to each type of mark) oreach type of mark can be specified by a sequence of marking mechanismactivations (e.g., pressing a sequence of pushbuttons). A joystick orwheel can also be implemented to enable specification of different marksin accordance with corresponding movement of the joystick or wheel (thiscan be particularly useful when the marks reflect different degrees ofsome quantity or quality). One or more menu displays can also be used toenable specification of different types of marks. In particular, suchmarking can be implemented so that the recorder is offered theopportunity to make a mark increasingly specific: an initial mark bringsup a menu display which allows the recorder either to accept the currentmarking definition or select a more specific marking definition,selection of a more specific marking definition brings up a menu displaywhich allows the recorder either to accept the current markingdefinition or select an even more specific marking definition, etc. (anydesired number of levels of such nesting can be used, though as apractical matter, it may be desirable to have no more than one, two orthree levels of increasing mark specificity).

As indicated above, the temporal vicinity of a mark can be defined asthe meaning or part of the meaning of a mark. When the control interfacedevice allows multiple marks, different marks can have a differenttemporal vicinity associated therewith. For example, one or more markscan specify marking of the data currently being recorded, while othermarks specify marking of data obtained a predetermined amount of timebefore or after the time of specification of the mark.

Multiple marks can also be used to assign differing levels of importanceor interest to the marked data. For example, one mark could indicatethat the marked data is extremely important or interesting, another markthat the marked data is very important or interesting, and a third markthat the marked data is moderately important or interesting. One or moremarks could also be used to indicate that the marked data isuninteresting or unimportant, so that the parts of the recordingcorresponding to such marks could be skipped during later playback ofthe recording.

Multiple marks can also be used to indicate a characteristic of thecontent which the marked data represents. For example, one mark could beused to indicate that the content represented by the marked recordingdata includes conversation, while another mark could be used to indicatethat the content represented by the marked recording data includes aparticular type of scene, while still another mark could be used toindicate that the content represented by the marked recording dataincludes a particular type of action.

Multiple marks can also be used to indicate the beginning and end ofactivity of interest during an event. As an illustration of the use ofsuch marks, during a recording of a birthday party, a recorder couldspecify a “start” mark when a birthday cake is being brought out and an“end” mark when the singing of “Happy Birthday” has been completed.Additional marks can also indicate a characteristic of the content ofthe marked activity of interest, as discussed above.

Multiple marks can also be used to specify information regarding therecording conditions. For example, different marks could be used toindicate the quality of the ambient lighting, ambient sound and/orfilming location. Different marks could also be used to indicate whethernoise is present in the audio recording or, further, the amount of noisepresent in the audio recording.

Multiple marks can also be used to specify a recorder's state of mindduring the recording. For example, different marks could be used toindicate different emotions (e.g., happiness, sadness, excitement,boredom)—and, perhaps, the degree of those emotions—experienced by therecorder during the recording. Physiological data that can be acquiredby a recording unit according to the invention, as discussed elsewhereherein, can also be used to implement marks of this type, thephysiological data being analyzed to infer emotion(s) of the recorder.

It can also be desirable to enable specification of one or more privacymarks, in addition to the other types of marks described above. (Herein,“privacy mark” refers to a mark that either restricts, prohibits oreliminates the possibility of access to the marked part of therecording, or indicates that it is desired to restrict, prohibit oreliminate the possibility of access to the marked part of therecording.) For example, one type of privacy mark could indicate thatthe marked part of the recording is personal. Such a mark may be used bya recorder to, for example, indicate that the recorder may want to takeappropriate steps after the event to inhibit the observation of thatpart of the recording by others. Another type of mark could indicatethat the marked part of the recording is to be erased. Such a mark maybe used by a recorder to, depending upon the particular implementationof the recording unit, cause the marked part of the recording to beerased during the event or indicate that the marked part of therecording is to be erased after the event.

Multiple marks can also be used to identify which of multiple recordingunits is known or thought to have acquired recording content ofparticular interest. As one illustration of the use of such marks, tworecorders may be using recording units according to the invention torecord content at different locations. (This marking capability may beparticularly useful when the recorders are at locations from which therecorders cannot make visual contact.) The recording units areimplemented to enable the recorders and/or the recording units tocommunicate with each other during recording so that each recorder isaware of the nature of the content being recorded by the recording unitof the other recorder. One of the recorders may decide, based upon suchcommunication, that some or all (audio recording, visual recording orboth) of the recording content being obtained by the recording unit ofthe other recorder is of particular interest, and specify a mark that soidentifies that recording unit. (The marking capability may be furtherimplemented to enable more specific identification of the recordingcontent of interest, e.g., audio recording or visual recording.)

Multiple marks can also be used to identify the person making the mark.For example, as described further below, a marking token that can bephysically separated from the control interface device of a recordingunit according to the invention can be used to specify one or moremarks. In particular, such a marking token can be adapted to enablewireless communication with the remainder of the recording unit, therebyenabling the marking token to be given to a person who possesses andcontrols only the marking token. Such use of a marking token can enablemultiple people to specify marks for the recording obtained by a singlerecording unit. Similarly, if the recording unit is adapted to enablecommunication with other recording units, it can be possible for thoseother recording units to communicate marks to the recording unit. Suchmarks are specified by persons other than those operating the recordingunit by which the marks will be stored. In either case, it can be usefulfor a mark to indicate the identity of the person specifying the mark.

Multiple marks can also be used to identify a person present in thecontent represented by the marked visual recording data. For example, itcan be desirable to specify a set of marks where each mark correspondsto one of some or all of the people participating in an event. Or,depending upon the nature of the event, it may be desirable to specify aset of marks where each mark identifies a person having a particularcharacteristic, i.e., a person belonging to a predefined group (e.g.,during a sports event, different marks can be used to identify membersof different teams participating in the event).

It can be particularly desirable to implement marking capabilities of arecording unit according to the invention so that the meaning of themarks available to a recorder can be varied (i.e., the meaningassociated with the activation of a particular marking mechanism orseries of marking mechanism activations can be changed). Such variablemark meaning can enable the recorder to specify particular desired markmeanings, prior to recording an event, that are related to the eventwhich is being recorded. For example, different marks could be definedto correspond to particular people that will participate in the event,so that activity or conversation by those persons during the event canbe denoted. Or, different marks could be defined to designate particularoccurrences during the event. For example, if the event is abirdwatching expedition, different marks could denote the sighting ofparticular types of birds.

Varying the meaning of marks can be accomplished, for example, byproviding such capability as part of a recording unit docking station(an apparatus separate from the recording unit) which is adapted toenable a user to transfer desired instructions and/or data (includingdata representing one or more marking definitions) to the data storagedevice(s) of a recording unit to effect particular operation of therecording unit. The docking station can include an interface whichenables a user to specify desired marking definitions. Such a dockingstation can also be implemented so that the docking station can beconnected to a network (e.g., a computer network such as the Internet)to enable marking definitions to be transferred to the recording unitfrom another network location via the docking station. A recording unitaccording to the invention can also be implemented so that one ofmultiple plug-in modules, each plug-in module including a data storagedevice (e.g., an EEPROM) having stored thereon one or more sets ofmarking definitions, can be engaged with the recording unit. Forexample, a theme park may provide multiple such plug-in modules, eachmodule including marking definitions for one or more attractions at thetheme park.

A recording unit according to the invention can also be implemented sothat the meaning of marks can be varied during recording of an event.Multiple sets of marking definitions can be stored on data storagedevice(s) of the recording unit, together with instructions regardingselection of a particular set of marking instructions in response eitherto particular user input or to content of data (recording data by thedata acquisition device(s) or data obtained by other device(s), such asa physiological monitoring device, biometric device or position sensingdevice) obtained by the recording unit. For example, the available typesof marks can vary with the position of the recording unit. The recordingunit can also be implemented so that the available types of marks varyover time during recording.

The marking capability (“augmented marking”) of a recording unitaccording to the invention can be implemented so that the specificationof a mark causes operation of the recording unit in addition to thestorage of the meaning and the time of specification of the mark. Forexample, specifying a mark that indicates that particular content is ofno interest can cause the recording unit to compress the recording dataacquired by the recording unit by a predefined amount that is greaterthan the amount of compression (if any) that occurs during normaloperation of the recording unit. Similarly, if multiple marks can bespecified indicating varying levels of interest, each such mark cancause the recording data to be compressed by a particular predefinedamount, e.g., as the mark indicates relatively less interest, thecompression ratio increases, and as the mark indicates relativelygreater interest, the compression ratio decreases. It may be desirablethat such marking capability be implemented so that the compression inaccordance with the mark ends after a predetermined amount of time(e.g., some amount of time less than about a minute), so that therecorder, in accordance with the goal of the invention that therecording unit be a low attention device, need not continue to monitoroperation of the recording unit to ensure that normal compression(again, if any) resumes when content of interest is again beingrecorded. Alternatively, such marking capability can be implemented sothat compression in accordance with the mark continues until aninstruction (such as a mark indicating content of interest) is specifiedby the recorder that causes such compression to stop. A recording unitaccording to the invention can also be implemented so that locations ofa data storage device at which recording data is stored that representscontent marked as being of no interest can be re-used to store newlyacquired recording data.

To facilitate the implementation of marking—in particular, to enablespecification of marks in a manner that requires as little attentionfrom the recorder as possible—it may be desirable to provide additionalapparatus (a “marking token”) that can be physically separated from therest of a recording unit according to the invention. For example, aring, brooch, bracelet (e.g., a watch-like band) or similar wearableitem can be constructed with one or more pushbuttons and adapted toenable communication (perhaps wirelessly) with the remainder of therecording unit. The recorder can then specify a mark by (almostreflexively) simply reaching for the marking token and depressing apushbutton. Each marking token can be adapted to enable specification ofone or more marks; however, it may be desirable, in keeping with thegoal of the invention that the recording unit be a low attention device,for each marking token to be confined to specification of a single typeof mark, in order to minimize the amount of attention that the recordermust devote to use of the marking token. A marking token can also beimplemented to enable storage of marking data thereon (i.e., can includea non-volatile data storage device), such a marking token furtherincluding a real-time clock that enables a correspondence between themarking data and the recording data stored by the recording unit to beestablished. A recording unit according to the invention can includemultiple marking tokens. A marking token that is adapted to enablewireless communication with the remainder of the recording unit can begiven to a person who possesses and controls only that part (the markingtoken) of the recording unit, thereby enabling that person to specifymarks during or after recording (either in addition to, or instead of,the person operating the remainder of the recording unit).

A recording unit according to the invention can also be implemented toenable a mark to be specified by a voice command (“voice marking”).Microphones that are also used to acquire audio recording data can beused for voice marking, for example.

In one implementation of voice marking, the recorder simply statesduring recording that particular recorded content is to be identified ina particular way. This mark can be used during later review of therecording.

In another implementation of voice marking, during recording therecorder speaks one or more of a particular set of “marking words” thatare recognized by the recording unit and used to generate appropriatemarking data. To enable such voice marking, a mechanism for indicatingthat a voice mark is to be imminently specified is employed to enable avoice mark to be understood (i.e., distinguished from audio recordingdata). Such mechanism can be, for example, a mechanical control (e.g.,pushbutton or switch) or a recognizable speech pattern or sound (e.g., apredetermined word or sequence of words). The identification of thespoken mark can then be determined using voice recognition techniques toanalyze the speech occurring after the indication of an impending mark(e.g., within a predetermined short duration of time after suchindication), as known to those skilled in the art.

The recording data can be analyzed to produce a mark or to supplement ormodify an existing mark. A mark can be produced based on analysis ofaudio recording data, visual recording data, or both.

For example, the audio recording data can be analyzed to determine ifspeech is present and, if so, the identity of the speaker. (Suchanalysis can be facilitated by identifying a set of possible speakersbefore beginning recording.) Recording data including speech can then bemarked with the identity of the speaker or speakers. Or, the audiorecording data can be analyzed to detect emphasis, cadence or othercharacteristics of speech present in the audio recording data. This canbe accomplished by, for example, analyzing the stress and/or timing ofaudio data within a set of audio recording data representing the speech.The characteristics of the speech can then be used to produce, forexample, marks indicating a probable level of interest of the recordedcontent (e.g., speech uttered rapidly or with a relatively large amountof stress tends to indicated excitement on the part of the speaker; therecording data obtained at that time may therefore be marked as probablyrepresenting content having a high level of interest). Techniques foraccomplishing the above-indicated types of audio data analysis are knownto those skilled in the art. For example, such techniques are describedin “The use of emphasis to automatically summarize a spoken discourse,”by F. R. Chen et al., 1992 IEEE International Conference on Acoustics,Speech and Signal Processing, vol. 1, pp. 229-232, San Francisco, 1992;“Construction and Evaluation of a Robust Multifeature Speech/MusicDiscriminator,” by Eric Scheirer et al., Proceeding of the 1998International Conference on Acoustics, Speech, and Signal Processing,Munich, 1997; “Baby Ears: A Recognition System for AffectiveVocalizations,” by Malcolm Slaney et al., Proceeding of the 1998International Conference on Acoustics, Speech, and Signal Processing,Seattle, 1998; and Advances in Speech Signal Processing, edited bySadaoki Furui and M. Mohan Sondhi, Chapter 22, pp. 701-738, 1992, thedisclosures of which are incorporated by reference herein. Techniquesfor accomplishing such audio data analysis are also described in thecommonly owned, co-pending U.S. patent application Ser. No. 08/760,769,entitled “Variable Rate Video Playback with Synchronized Audio,” by NealA. Bhadkamkar et al., filed on Dec. 5, 1996, the disclosure of which isincorporated by reference herein.

The visual recording data can be analyzed to identify colors, textures(i.e., gradations in visual pattern) and/or shapes which can, in turn,be used to identify objects in the visual recording content. Thecorresponding recording data can then be marked with a mark appropriateto the visual recording content. For example, analysis of the visualrecording data may indicate the presence of particular scenery (e.g., abody of water, a tree); a corresponding mark can be used to indicate thepresence of that scenery in the recording data. Or, for example, thevisual recording data can be analyzed to identify the presence of aperson in the visual recording content and, if a person is present, thevisual recording data can further be analyzed to identify the personbased on facial or other features. Techniques for accomplishing theabove-indicated types of visual data analysis are known to those skilledin the art. For example, such techniques are described in “Searching forDigital Pictures,” by David Forsyth et al., Scientific American, June1997; Face Recognition: From Theory to Applications, edited by HarryWechsler et al., pp. 244-261, 1998; “Face and feature finding for a facerecognition system,” by Andrew W. Senior, Proceedings of the SecondInternational Conference on Audio- and Video-based Biometric PersonAuthentication, Washington, D.C., 1999; and “Multi-Sensor BiometricPerson Recognition in an Access Control System,” by Bernhard Froba etal., Proceedings of the Second International Conference on Audio- andVideo-based Biometric Person Authentication, Washington, D.C., 1999, thedisclosures of which are incorporated by reference herein.

As discussed elsewhere herein, a recording unit according to theinvention can be implemented to enable acquisition of data other thanrecording data or marking data. In particular, the recording unit can beimplemented so that such data can be used to supplement or modifyanother mark in some way, or so that such data can be used as a markitself. For example, a recording unit according to the invention caninclude one or more physiological monitoring devices that obtain dataregarding a physical characteristic of the recorder, one or morebiometric devices, and/or one or more position sensing devices, each ofwhich can acquire data that can be used in the above-described way.Since a non-contemporaneous mark will typically be specified with somedegree of uncertainty, such data can advantageously be used, inparticular, to assist in the accurate identification of recordingcontent intended to correspond to a non-contemporaneous mark. Further,the importance of using such data for this purpose increases with thedistance in time between the marked time and the marking time at whichthe mark is specified. Particular ways of using such data to produce amark or to supplement or modify another mark are described in moredetail in the above-referenced commonly owned, co-pending United StatesPatent Application entitled “Flexible Marking of Recording Data by aRecording Unit.”

The marking capabilities described herein, while advantageous whenimplemented as part of a recording unit according to the invention or aspart of post-event processing of recording data obtained by a recordingunit according to the invention, can, more generally, be implemented aspart of any recording apparatus or method, or apparatus or method forprocessing recording data.

As indicated above, a recording unit according to the invention can beimplemented so that recording data that has been previously obtained andstored by the recording unit can be retrieved and used to generate adisplay of a previously acquired part of the recording. The controlinterface device can include mechanisms that enable the recorder tospecify one or more instructions that cause such recording display tobegin and to terminate. The control interface device can also includemechanisms that speed up or slow down the display, or that cause thedisplay to scan or skip backward or forward in time. Identification ofthe desired part of the stored recording data can be accomplished in anyof a variety of ways. In general, the specification of the part of therecording to be displayed can be done in ways similar to those describedabove with respect to non-contemporaneous marking. For example, therecorder can specify an amount of time prior to the current time, anabsolute time, or a particular type of mark. The recording display canthen begin in the temporal vicinity of recording data corresponding tothe specification made by the recorder, the definition of temporalvicinity being a pre-established characteristic of the recording unit orcapable of specification by the recorder. As other mechanisms of acontrol interface device of a recording unit according to the invention,such mechanism can be implemented using any of a variety of well-knownmechanisms (e.g., pushbuttons) that are appropriately connected to theelectronic devices (e.g., system controller) of the recording unit toeffect such control, as known to those skilled in the art ofconstructing user interface devices.

The control interface device can also include mechanisms that enablesearching of previously recorded, stored recording data. (Such searchingwould typically be accompanied by display of an identified section ofthe stored recording, as described above.) Searching can be implementedin a variety of ways. For example, a search for a particular type ofmark or marks can be performed. Searching for previously recordedcontent of interest can also be accomplished by specifying display of aparticular part of the recording (e.g., the section of the recordingacquired a specified duration of time prior to the present), asdiscussed above. Display speed control, as also discussed above, canalso be used to facilitate searching for desired content.

Above, examples have been given of various types of control functionsthat can be implemented in a control interface device of a recordingunit according to the invention. It is to be understood that there aremany other possibilities; in general, the control interface device canbe implemented so as to allow any type of control that is possible withthe devices that comprise the recording unit.

A control interface device in a recording unit according to theinvention can also include a control interface display for enabling therecorder to input control instructions to the recording unit and/or topresent information to the recorder regarding operation of the recordingunit. In general, the control interface display can include a visualdisplay and/or an audio display.

The LED(s) used to indicate whether a recording unit is recording ornot, whether recording data is being transmitted or not, and whether therecording unit is powered or not, described above, are examples of theprovision of a visual display as part of a control interface display.One or more LEDs can also be included in a recording unit according tothe invention to indicate other binary information (e.g., systemoperation normal or malfunctioning).

A text display can also be included as part of a control interfacedisplay. The text display can be used, for example, to conveyinstructions to the recorder regarding some aspect of the operation ofthe recording unit or to provide feedback to the recorder regarding theoperational status or performance of a component of the recording unit.In keeping with the goal of making the recording unit small andunobtrusive, it may be desirable to make the text display relativelysmall. As a result, the display may allow only a small amount of text tobe displayed at one time (e.g., one, two or several lines, and arelatively small number of characters per line). To facilitate thepresentation of information on a small text display, it may be desirableto implement the display so that characters scroll across the display.The use of a larger text display enables more text to be presented atone time (perhaps avoiding the need for scrolling text, which can bedifficult to read) and/or a more legible display of the text. However,as indicated above, the larger display may be more be obtrusive andheavier. Whether a large or small display is used can depend onconsideration of these tradeoffs in view of the intended application(s)of the recording unit.

A control interface display can also include a display that makes use ofgraphical user interface (GUI) techniques, as known by those skilled inthat art. For example, appropriate GUI display mechanisms (e.g.,graphical pushbuttons, slider bars, data entry boxes, checkboxes, bulletcircles, etc.) can be provided that allow the recorder to specifyinstructions regarding any of the above-described control functions.Other appropriate GUI display mechanisms (e.g., checkboxes, bulletcircles, flashing or color-coded parts of the display) can be used toprovide information regarding operation of the recording unit. However,the capability of providing such GUI display mechanisms may be limitedby the desire to keep the display relatively small.

Like the visual recording display device, it can be advantageous to usean LCD display to implement a control interface visual display becauseof the ready availability and relatively low cost of LCD displays.

The control interface visual display can be integrated in a singledevice with a visual recording display or can be implemented in aseparate device. One way to integrate the control interface visualdisplay and the visual recording display is to provide those displaysseparately (e.g., in separate display windows or on separate displayscreens) in a single display device. Another way to integrate thecontrol interface visual display and visual recording display is tocause the control interface visual display to be overlaid on the visualrecording display (e.g., the control interface visual display could bedisplayed near the bottom or other edge of the visual recordingdisplay). It may also be desirable, for example, to overlay only a partof the control interface visual display on the visual recording display,the remainder of the control interface visual display being provided bya separate display device. In particular, overlaying a control interfacetext display on a visual recording display can be done in a manner thatdoes not overly detract from the visual recording display. Overlaying acontrol interface text display on a visual recording display requiresthe provision of additional driver electronics, as known to thoseskilled in the art. Overlaying a control interface text display on avisual recording display can be a particularly advantageous way ofintegrating a control interface display and a visual recording displayinto a single display.

An audio display of a control interface device can be used to conveyinformation to a recorder as described above with respect to the controlinterface visual display. The use of a control interface audio displaycan be advantageous because it allows information regarding theoperation of the recording unit to be conveyed to the recorder in amanner that requires no visual attention from the recorder.

The control interface audio display is typically a speaker that isincluded integrally with other parts of the control interface device.The control interface audio display can also be formed integrally withthe audio recording display device. In particular, the two audiodisplays can be implemented by the same device. For example, a headsetor a pair of speakers mounted near the recorder's head (one on eachshoulder) can be used as both the control interface audio display andaudio recording display device. Audio input to the headset or speakerscould be controlled, for example, so that the audio recording is emittedfrom one speaker of the headset or one of the shoulder speakers, andcontrol interface audio output is emitted from the other speaker of theheadset or the other shoulder speaker. Or, the control interface audiooutput could simply be overlaid on to the audio recording, both beingemitted from both speakers of the headset or both shoulder speakers, thecontrol interface audio output being given a distinctive sound or soundsso that it can be unambiguously identified by the recorder. Or, theaudio display device can be implemented to display only one of the audiorecording display or control interface audio display at a time, thecontrol interface device including a control that enables the recorderto switch between the two audio displays.

The control interface audio display can be controlled to emit one ormore tones to communicate information regarding the operation of therecording unit. For example, a control interface audio display could beimplemented so that a single tone of specified duration indicates therecording unit has been turned off. Two tones could indicate that therecording unit has been turned on. Different types, sequences and/orpitches of tones can be used to convey any of a variety of informationregarding operation of the recording unit, in particular informationregarding the control functions described above.

A more sophisticated control interface audio display could make use of adigitized voice to report information regarding the operation of therecording unit. Such capability can readily be implemented by one ofskill in the art.

Above, the use of a control interface audio display to reportinformation to the recorder is described. An audio data acquisitiondevice could also be included as part of a control interface device of arecording unit according to the invention, so that the recorder caninput control instructions to the control interface device via spokencommands. The audio data acquisition device itself can be a conventionalmicrophone. However, when an audio data acquisition device is includedas part of the control interface, the recording unit will also have toinclude appropriate hardware and/or software for identifying the spokencommands and filtering out background noise. Such filtering can beaccomplished using methods and/or apparatus known to those skilled inthe art.

FIG. 5 illustrates a control interface device 500 according to anembodiment of the invention.

A power button 501 is used to turn power to the recording unitcomponents off and on. Preferably, a power button of a control interfacedevice of a recording unit according to the invention is located and/orconstructed so as to reduce the possibility of being activatedaccidentally. For example, the power button 501 can be implemented sothat a relatively large force must be exerted to activate the powerbutton 501.

A control interface visual display 502 enables display of information tothe recorder, in particular, information regarding control and operationof the recording unit. For example, the control interface visual display502 can display control options in response to activation of buttons ofthe control interface device 500. The control interface visual display502 can be, for example, a two-line, 12 character LCD display. Thebuttons 502 a and 502 b enable the recorder to scroll through thecontrol interface visual display 502. The buttons 502 c and 502 d enablethe recorder to select one of two displayed control options: the buttons502 c and 502 d are positioned to underlie corresponding control optionsdisplayed on the control interface visual display 502.

A visual recording display 503 enables display of a recording to therecorder. In response to appropriate instruction from the recorder, asdescribed further below, the visual recording display 503 can displaythe content currently being recorded or a part of a recording that haspreviously been recorded. The visual recording display 503 can be, forexample, a Casio VM-50 LCD TFT display.

The LEDs 511 a and 511 b indicate operational status of the recordingunit. For example, one of the LEDs 511 a and 511 b (the “system statusindicator”) can be used to indicate system status information, such aswhether the recording unit is powered or not and whether the recordingunit is operating normally or malfunctioning in some manner.Illustratively, the system status indicator can be continuouslyilluminated to indicate that the recording unit is powered and operatingnormally, and the system status indicator can flash to indicate a systemmalfunction or to indicate the recording unit is powering up or poweringdown. The other of the LEDs 511 a and 511 b (the “recording/transmissionstatus indicator”) can be used, for example, to indicate recording andtransmission status. Illustratively, the recording/transmission statusindicator can be continuously illuminated to indicate that the recordingunit is recording and not transmitting recording data, therecording/transmission status indicator can flash slowly to indicatethat the recording unit is recording and transmitting recording data,and the recording/transmission status indicator unit can flash rapidlyto indicate that the recording unit is not recording but is transmittingrecording data.

The button 507 enables the recorder to start and stop recording, as wellas to start and stop transmission of the recording to other recordingunits. The button 507 is preferably implemented so that a relativelylarge force must be exerted to activate the button 507. Activation ofthe button 507 causes the control interface visual display 502 todisplay control options to the recorder that enable the recorder tostart or stop recording, and to start or stop transmission of recordingdata.

The buttons 504 a, 504 b, 504 c and 504 d enable control of the displayof a recording by the visual recording display 503. The button 504 acauses display of a recording that moves forward in time, while thebutton 504 b causes display of a recording that moves backward in time.The button 504 c causes display of a recording to stop. The button 504 dcauses the display of a recording to skip in a predetermined manner inthe temporal direction in which the recording was being displayed. Eachof the buttons 504 a, 504 b, 504 c and 504 d can also be implemented sothat depressing the button 504 a, 504 b, 504 c or 504 d in differentmanners causes different operation of the visual recording display 503.For example, the button 504 d can be operated in different modes: onemode can cause the display of the recording to skip to the next mark,while other mode(s) can cause the display of the recording to skip by aspecified amount of time. The buttons 504 a, 504 b and 504 c candesirably be shaped as shown in FIG. 5, in accordance with conventionalgraphical representations of the functionality embodied by thosebuttons, as used in other user interfaces for recording and displaydevices.

The button 508 enables the recorder to review a part of a recording thathas previously been recorded. The button 508 is preferably implementedso that a relatively large force must be exerted to activate the button508. Activation of the button 508 causes the control interface visualdisplay 502 to display control options to the recorder that enable therecorder to specify the point at which to begin the display ofpreviously recorded content. After activation of the button 508, thebuttons 504 a, 504 b, 504 c or 504 d can be used as described above tonavigate through the recording.

The button 510 enables the recorder to change the recording beingdisplayed by the visual recording display 503. Activation of the button510 can cause display of an identification of the recording currentlybeing displayed, as well as control options that enable the recorder toselect a particular recording for display.

The button 509 enables the recorder to check the operational status ofcomponents of the recording unit (e.g., remaining battery power, datastorage capacity used and/or available) and to effect changes inoperating parameters (e.g., compression ratio). The button 509 ispreferably implemented so that a relatively large force must be exertedto activate the button 509. Activation of the button 509 causes thecontrol interface visual display 502 to display control options to therecorder that enable the recorder to accomplish such functions.

Marking buttons 505 a, 505 b and 505 c enable specification of differenttypes of marks. For example, each button 505 a, 505 b and 505 c can bededicated to a particular type of mark. Alternatively, one or more ofthe marking buttons 505 a, 505 b and 505 c can enable specification ofmultiple types of marks, each such button 505 a, 505 b and/or 505 ccausing display of control options by the control interface visualdisplay 502 which allow selection of particular types of marks. (Ingeneral, such mark selection control options can be nested to anydesired degree and can be implemented so that the recorder can select amark from any level of the hierarchy.) Each of the buttons 505 a, 505 band 505 c has a unique shape, so that the recorder can operate a desiredone of the buttons 505 a, 505 b and 505 c with little or no visualobservation of the buttons 505 a, 505 b and 505 c, in accordance withthe goal of the invention that a recording unit be a low attentiondevice.

The button 506 provides the functionality of a kill switch, as discussedabove. Preferably, a kill switch of a control interface device of arecording unit according to the, invention is located and/or constructedso as to reduce the possibility of being activated accidentally. Forexample, the button 506 is located near one edge of the controlinterface device 500, i.e., out of the “mainstream” of recorderinteraction with the control interface device 500. Additionally, thebutton 506 can be implemented so that a relatively large force must beexerted to activate the button 506. However, the button 506 ispreferably made relatively large to facilitate quick and reliableactivation of the button 506 by the recorder.

The color, shape, location (including grouping) and/or texture ofbuttons of the control interface device 500 can be specified tofacilitate interaction by the recorder with the control interface device500, thus facilitating low attention recording. For example, relatedbuttons (e.g., the marking buttons 505 a, 505 b and 505 c) can be madethe same color, shape and/or texture, which can be a different color,shape and/or texture than any of the other buttons of the controlinterface device 500. The color, shape and/or texture of a button canalso be specified so as to convey a sense of the function of the button,e.g., the button 506 can be made red (suggesting stopping).

8. Position Sensing Device

A position sensing device can be included in a recording unit accordingto the invention to enable the position of the recording unit to beascertained at any point in time. (As used herein, a “position sensingdevice” can include one or more devices that enable the position, asdefined below, of a recording unit to be determined.) Data representingthe position can be stored in a manner that relates the position data tocorresponding recording data obtained at the same time.

The “position” of a recording unit includes the location of therecording unit and/or the orientation of the recording unit. Thelocation can be specified in global coordinates (i.e., relative toearth), local coordinates (i.e., relative to a fixed region or object onearth), or relative coordinates (e.g., relative to other recordingunits). The orientation is typically defined with respect to the filmingorientation of the visual data acquisition device (which can bespecified with respect to one or more of three perpendicular rotationalaxes, i.e., pan, tilt and rotational axes, as known to those skilled inthe art). The orientation can also be defined with respect to anotherpart of the recording unit or a part of the recorder's body.

Whether location is specified in global, local or relative coordinatescan depend on the implementation of the position sensing device, theintended environments within which the recording unit is to be usedand/or the intended use of the position information. For example, whenthe position sensing device is a GPS receiver, as described furtherbelow, location is specified in global coordinates. If the recordingunit is to be used in an environment in which GPS signals cannot bereceived by the recording unit, as discussed below, then a positionsensing device that relates the recording unit location to nearbyobjects or to other recording units may be used, the former specifyinglocation in local coordinates and the latter in relative coordinates. Ifthe position information is to be used to spatially integrate recordingsafter the event, then it can be advantageous to use a position sensingdevice that specifies location relative to other recording unit(s).

The position information can be further used to determine relatedderivative information, such as, for example, a path of movement of therecording unit or the velocity at which the recording unit is moving.(If a GPS system is used velocity information is provided directly bythe GPS system.) The position information and related derivativeinformation can be used, for example, to augment the display of therecording (either in real time as part of the display of the recordingby a recording unit, or as part of post-event display of the recording)or to facilitate integration of the recording obtained by the recordingunit with the recording(s) obtained by other recording unit(s) (to, forexample, enable an integrated post-event display of multiple recordingsof an event).

A global positioning system (GPS) receiver can be used, for example, toembody a position sensing device of a recording unit according to theinvention that enables determination of the location of the recordingunit. Either standard or differential GPS can be used. (Differential GPSnecessitates the use of a second receiver.) The latter is more accurate,but is also more expensive and may not be available in many areas.Currently, standard GPS provides a location with an error of less thanor equal to about 30-50 feet. However, in the near future, the maximumerror of a GPS location determination will decrease to about 5-10 feet,when certain planned modifications are made to the implementation ofstandard GPS. Further, there are existing private satellite servicesthat can be used to improve the accuracy of GPS location determinations(this undesirably adds to the cost of the recording unit, however, dueto the need for another, or a more complex, receiver, and to the need topay a subscription fee to access the private satellite service'ssignal). In any event, any error in location is typically the same forall recording units being used at an event, so that the relativelocations of the recording units can be known with a high degree ofaccuracy.

If the event is one that takes place in a building or other environment(such as a high mountain or a city with tall buildings) in which therecorder may be shielded from GPS satellites, the GPS receiver(s) willbe ineffective to establish the location of the recording unit. In sucha situation, it is necessary to make use of an alternative positionsensing system if such information is necessary or desirable.

A GPS receiver can also be used to receive a signal representing thecurrent time that can be used as a clock to generate time-stamps for therecording data. Depending upon the speed of the processing device(s) ofthe recording unit, the GPS time signal can be updated about once asecond. The GPS time signals are extremely accurate; thus, since all ofthe recording units are receiving this time signal, the time-stamps ofall recording units are adequately synchronized.

Though the recording unit must not be constantly blocked from GPSsatellites in order for the recording unit to receive such time signals,the recording unit need only be exposed to a GPS satellite for about aminute to enable the time signals to begin. Further, after that point,clock drift occurs slowly, even if the recording unit is blocked fromthe GPS satellites for a period of time. Additionally, a recording unitaccording to the invention typically also includes an internal clock, asdescribed above, that, while not as accurate as the GPS time signals,can also be used to accurately time-stamp data.

Other location tracking systems can also be used in a recording unitaccording to the invention. For example, local positioning systems canbe used in which transmitters or receivers are placed beforehand atvarious locations in the area within which the event will occur, and acorresponding receiver or transmitter, respectively, is included as partof recording units according to the invention. (Such systems are similarin operation to the GPS-fix system described above.) However, such localpositioning systems may be disadvantageous in that the recording unit isconstrained to use within the area of operability of the system.Further, setting up, configuring and disassembling such a system foreach event location may be undesirably burdensome.

Transmitters and receivers that are part of multiple recording units canbe used to implement a relative positioning system. The use of such apositioning system can be advantageous because such transmitters andreceivers may already be included as part of the recording units forother purposes (e.g., to transmit recording data signals betweenrecording units and/or to enable communication between recorders). Therelative positions of the recording units can be determined frombroadcasted signals using methods known to those skilled in the art.

An electronic compass can be used, for example, to embody a-positionsensing device of a recording unit according to the invention thatenables determination of the orientation of the recording unit.Electronic compasses are typically accurate within about 1 or 2 degrees.Electronic compasses that compensate for interference due to thepresence of ferro-magnetic material (in particular, ferro-magneticmaterial that may be present in the recording unit) can be used with arecording unit according to the invention. Typically, as indicatedabove, the orientation of the recording unit is defined with respect tothe filming orientation of the visual data acquisition device. Thus, itis desirable to attach the compass (or other orientation sensing device)to the visual data acquisition device so that the sensed orientation canbe easily related to the filming orientation. However, the compass canbe mounted on the recorder in other ways so long as the relationshipbetween the orientation of the compass and the visual data acquisitiondevice is known.

Other devices or systems can be used to determine the orientation of therecording unit. For example, an inertial reference system using one ormore gyroscopes can be used. A tilt sensor can also be used. Inparticular, a tilt sensor that can compensate for acceleration can beadvantageously used.

It may be desirable to use multiple location tracking systems and/ormultiple orientation tracking systems in a recording unit according tothe invention to increase the accuracy of the position determination.However, this can add to the cost of the recording unit. The number andtype of position tracking systems used in a recording unit according tothe invention can be chosen in view of such considerations.

9. Communication Among Components of a Recording Unit

In general, communication among the components of a recording unitaccording to the invention can be implemented using conventionalapparatus and techniques (e.g., using conventional bus techniques andapparatus), and is controlled or mediated by the system controller.

It may be necessary or desirable to implement the recording unit so thatcomponents of the recording unit are physically separate. For example, arecording unit may include a video camera mounted on the shoulder of therecorder, a visual recording display device mounted on the recorder'swrist, and other components (e.g., the system controller, data storagedevice and data processing device) positioned within an encasementstrapped to the recorder's back. In general, communication betweenphysically separate components of a recording unit can be implementedusing either wired or wireless communication techniques and apparatus.The implementation of wired and wireless communication techniques andapparatus is known to those skilled in the art.

It may be particularly desirable to implement the recording unit so thatcomponents of the recording unit that are physically separate (or, atleast, components that are most physically separate, or physicallyseparate more than a specified distance) communicate with each other viawireless communications, since then there will be no wires that mayobstruct or distract the recorder. Additionally, if the visual dataacquisition device and/or the audio data acquisition device are notconnected to other components of the recording unit with wires, there ismuch greater flexibility in deciding where to position the dataacquisition devices and/or the other components of the recording unit.For example, if the recorder is riding on a bicycle while recording, avisual data acquisition device that communicates with the rest of therecording unit via wireless communication may be able to be located onthe bicycle while the part of the recording unit with which the videocamera communicates is located with other parts of the recording unit ina backpack worn on the recorder's back. Alternatively, the opposite maybe desirable: the visual data acquisition device can be mounted on therecorder, while the rest of the recording unit is mounted on the bicycle(this may be desirable to reduce the weight carried by the recorder,thus increasing the recorder's comfort, consistent with the goal of theinvention to make recording as “painless” as possible). If the visualdata acquisition device is connected to the rest of the recording unitwith a wire, these configurations may not be possible, or it may benecessary to position the part of the recording unit connected by wireto the visual data acquisition device on the bicycle or the recordernear the visual data acquisition device. As can be appreciated, theremay be many other circumstances in which it is desirable to be able tomove the visual or audio data acquisition device to a position that isrelatively distant from the part of the recording unit with which thevisual or audio data acquisition device communicates; wiredcommunication may prevent such movement or, at best, result in thepresence of wires which interfere with or distract the recorder. Forexample, a recording unit can be used by a recorder who is water skiing:for use in recording such activity, the visual data acquisition device(and, perhaps, audio data acquisition device) can be worn by therecorder, while the rest of the recording unit is positioned on, forexample, the boat. Or, for example, the recorder may desire to haveanother person take the visual data acquisition device and obtain, for aperiod of time, a visual recording. This may be desirable, for example,to enable the acquisition of visual recording data including therecorder. This may also be desirable to enable a small child to obtainrecording data without committing an entire recording unit to the child(an entire recording unit may be undesirably heavy for a child or may betoo expensive to entrust to a child). Similar flexibility in positioningcan also be desirable for other components of a recording unit, such asthe control interface device and recording display devices.

If wired communication is used, it is desirable to ensure that the wiresdo not interfere with the recorder. The accomplishment of this goal canbe facilitated by, for example, minimizing the distance between separatecomponents and/or otherwise controlling the position of separatecomponents of the recording unit. For example, extending acommunications wire between a video camera positioned on a recorder'sshoulder and other system components positioned within an encasementstrapped to the recorder's back can be done with little or noobstruction or distraction of the recorder. Additionally, this goal canbe attained by using coiled cable (similar to that used for telephonehandsets) for the connection between components, the cable extending arelatively short distance and tending to remain untangled when the cableis not fully extended.

It can be desirable to implement a recording unit according to theinvention so that additional devices can be easily added to therecording unit. For example, a recording unit according to the inventioncan be provided with, using methods and apparatus known to those skilledin the art, capability for connecting a device to the recording unit viaone or more serial ports or a bus protocol, such as the universal serialbus (USB), IEEE 1394 (e.g., Firewire or iLink) and/or IIC (i.e., I²C)protocols. As with other components of a recording unit according to theinvention, interaction of such device(s) with the rest of the recordingunit is controlled or mediated by the system controller, though it maybe necessary or desirable to provide a special purpose processor tofacilitate and/or enable such interaction. The use of the USB or I²Cprotocols may be desirable because those protocols are currentlyrelatively less expensive to implement than Firewire. The use of the I²Cprotocol may be desirable because that protocol requires a relativelysmaller processing capability than that needed for USB or Firewire.

10. Packaging of Recording Unit Components

In general, it is desirable to “package” the recording unit componentsin any manner that achieves some or all of the desired characteristicsof a recording unit according to the invention, e.g., small size, lowweight, unobtrusiveness, and capability of being mounted on a recorder(or, at least, capability of being easily carried by the recorder).

For example, to achieve the goal of making the recording, unit small, itcan be desirable to include many of the components of the recording unit(e.g., components other than data acquisition device(s), displaydevice(s) and control interface device(s)) together as part of a singleapparatus (e.g., enclose the components within an encasement), so thatthe recording unit can be made as compact as possible. (It may not bedesirable to include the data acquisition device(s), display device(s)and/or control interface device(s) in a single apparatus together withthe other components of the recording unit, since it is often desirableto adapt one or more of those devices so that they can be mounted atdifferent locations on the recorder.) Additionally, enclosing thosecomponents within a flexible encasement can be advantageous, since theflexibility may make the recording unit more comfortable to wear and mayenable the recording unit to be made even more compact. Includingcomponents together as part of a single apparatus may also facilitatemaking the recording unit as lightweight as possible.

On the other hand, it may be desirable to separate the recording unitcomponents into several parts. This can be desirable to enable the partsof the recording unit (and, thus, the weight of the recording unit) tobe more evenly distributed over the recorder's body. This can also bedesirable to reduce the bulkiness of any one part of the recording unit.For example, the power supply, which is often a relatively bulky andheavy component of a recording unit according to the invention, canadvantageously be separated from other components of the recording unit.However, if wired communication between recording unit components isused, the more that recording unit components are separated from eachother, the greater number of interconnecting wires that must be used.Wires can interfere with the recorder's movement. Further, the wires aresusceptible to damage from movement by the recorder and/or contact withother objects.

Further, it is desirable that the recording unit components be packagedso that the recording unit has a discreet appearance, rather than anostentatious or “technological” one (e.g., a “cyborg” look). This is inkeeping with the goal of making the recording unit unobtrusive(particularly to persons other than the recorder) so that the recordingunit can best be used for its intended purpose, i.e., as a device thatenables low attention recording. For example, it can be desirable tointegrate components of a recording unit according to the invention intoclothing to be worn by a recorder. In particular, small components of arecording unit according to the invention (e.g., microphones, GPSreceiver, video camera) can be sewn into clothing. Illustratively, aspecial police officer's uniform or a rental tuxedo may have sewntherein components of a recording unit according to the invention.

It can also be desirable that the recording unit components be packagedso that parts of the recording unit that come into contact with therecorder or that are susceptible to becoming dirty can be cleaned. Forexample, those parts of the recording unit can be adapted to beseparated from the rest of the recording unit to enable such cleaning.Those parts of the recording unit can also be constructed of materialsthat are capable of being cleaned without damaging the functionality ofthe recording unit and that are, preferably, relatively easy to clean.For example, those parts of the recording unit can be constructed sothat they do not include electronics or cables that would be damaged bycontact with water or a cleaning fluid.

FIG. 6 illustrates a recording unit according to the invention. Most ofthe components of the recording unit are enclosed within a backpack 601that is worn by a recorder 602. A visual data acquisition device 603 ismounted on a strap of the backpack 601 at about the recorder's chestlevel. A control interface device 604 can be carried by the recorder 602and is sized so that, when not in use, the recorder 602 can carry thecontrol interface device 604 in a pocket of the recorder's clothing. Thevisual data acquisition device 603 and control interface device 604 areconnected by wires 605 a and 605 b to the components in the backpack601, the wires 605 a and 605 b passing through a hole formed in thebackpack 601. The backpack straps can be adjusted to enable therecording unit to be comfortably and securely worn by people have avariety of body sizes and shapes.

Illustratively, a recording unit as shown in FIG. 6 can be implementedusing off-the-shelf components and packaging materials so that therecording unit weighs about 24 pounds. It is anticipated that arecording unit implemented using components that are particularlydesigned for use in the recording unit can be implemented so that therecording unit is lighter (and smaller): such a recording unit may weighabout as much as a conventional camcorder and be of comparable size.

11. Specialized Recording Unit Implementations

a. Recording Unit for Use by Vigorously Active Recorder

It is anticipated that a recording unit according to the invention willsometimes be used for recording an event during which the recorderengages in vigorous physical activity. For example, a recorder maydesire to obtain a recording while engaging in an athletic activity,such as hiking, snow skiing or ping pong. Such events particularlydemand that the recording unit allow the recorder freedom of movementand not distract or encumber the recorder's participation in theactivity. It can be desirable to tailor a recording unit according tothe invention for use in recording such events, with an increasedemphasis on the wearability characteristics of the recording unit (inparticular, vis-a-vis vigorous recorder movement) as compared to otheraspects of the recording unit.

FIG. 7 is a block diagram of the functional components of a recordingunit 700 according to a particular embodiment of the invention that is,as will be better appreciated from the description below, particularlywell suited for use in recording an event during which the recorderengages in vigorous physical activity. In general, except as modified bythe description below, aspects of the recording unit 700 can beimplemented as described above.

As illustrated in FIG. 7, the recording unit 700 includes threephysically separate components: a data acquisition unit 701, a controlinterface unit 702, and a data storage and system control unit 703. (Arecording unit according to the invention such as the recording unit700, i.e., for use by a recorder engaging in vigorous physical activity,can be separated into a different number—greater or fewer—of components.Some examples of such modification of a recording unit are discussedbelow; it will be understood that other such modifications can be madein accordance with the desired characteristics of this type of recordingunit according to the invention, as described elsewhere herein.)Separating a recording unit into physically separate components enablesthe weight of the recording unit to be distributed over the recorder'sbody to enable the recorder to more easily and comfortably support therecording unit, and to reduce the effect of the recording unit on therecorder's balance. Further, each component can be made relativelysmaller than a single device that embodies all of the functionality ofthe recording unit 700, thus reducing the propensity of components topull away from the recorder's body and reducing the likelihood of injuryif the recorder falls on a component. Additionally, each component canbe mounted on a recorder at a location that is particularly desirablefor that component. For example, components with which the recorderinteracts during recording can be mounted at relatively accessiblelocations, while components with which the recorder interacts little ornot at all can be mounted on locations that are less accessible. Each ofthese characteristics is particularly desirable in a recording unit thatis to be used for recording an event during which the recorder engagesin vigorous physical activity.

The data acquisition unit 701 includes a visual data acquisition devicefor obtaining visual recording data and an audio data acquisition devicefor obtaining audio recording data. In view of the anticipated use ofthe recording unit 700, it is particularly desirable that the visualdata acquisition device and audio data acquisition device be small andlightweight.

For example, a “board” camera can be used as a visual data acquisitiondevice of a recording unit according to the invention such as therecording unit 700. Board cameras include, for example, camerasconstructed using charge-coupled device (CCD) technology (the ChugaiCEC-100, discussed above, is an example of such a camera) and camerasconstructed using CMOS technology (referred to herein as a “CMOScamera”). Contemporaneous advances in technology are producing boardcameras that include a single chip mounted on a small (e.g., less thanone inch square) circuit board; such cameras can advantageously be usedin a recording unit according to the invention such as the recordingunit 700.

The audio data acquisition device can be embodied, for example, by twoomni-directional stereo lavalier microphones (e.g., Lectret 5200microphones, discussed above). It is desirable to mount the twomicrophones symmetrically on the front of the recorder's body. Forexample, when the recording unit 700 is embodied as shown in FIGS. 8A-8C(discussed below), one of the microphones can be mounted on eachshoulder strap of a harness used to mount the recording unit 700 on arecorder. The microphones are preferably mounted on the front of therecorder's body so that the recorder's body helps to shield themicrophones from sounds emanating from behind the recorder (which areexpected to be unwanted or relatively unimportant).

The audio data acquisition device can also be embodied by twodirectional microphones. Again, it is desirable to mount the twomicrophones symmetrically on the recorder's body: when the recordingunit 700 is embodied as shown in FIGS. 8A-8C (discussed below), one ofthe microphones can be mounted on each shoulder strap of a harness usedto mount the recording unit 700 on a recorder. However, directionalmicrophones need not be mounted on the front of the recorder's body; thedirectional microphones can be mounted on the straps adjacent to thetops of the recorder's shoulders.

The data acquisition unit 701 also includes mechanisms for mounting theaudio and visual data acquisition devices on the recorder, anotherperson or another mounting object. In general, mounting mechanisms asdescribed elsewhere herein can be used. It can be advantageous to mountthe audio and/or visual data acquisition devices so that the audioand/or visual data acquisition devices are detachably attached. It canbe particularly advantageous to detachably attach the visual dataacquisition device so that the visual data acquisition device can bemounted at various locations and/or so that the visual data acquisitiondevice can be handheld.

A mounting mechanism as described in the U.S. patent application Ser.No. 09/225,993, discussed above, can advantageously be used to mount avisual data acquisition device of a recording unit according to theinvention that is to be used for recording an event during which therecorder engages in vigorous physical activity. That mounting mechanismenables a visual data acquisition device to be mounted on a strap sothat the visual data acquisition device can be easily moved to differentlocations on the strap. That mounting mechanism also enables the filmingorientation of the visual data acquisition device to be easily changedabout any of three perpendicular axes. Advantageously, that mountingmechanism enables multiple positioning parameters (e.g., orientationabout multiple axes) to be adjusted with one hand and, in some cases,using a single control mechanism of particular importance for arecording unit (such as the recording unit 700) that is intended to beused to record events during which the recorder engages in vigorousphysical activity, that mounting mechanism also attaches the visual dataacquisition device to a person very stably.

The visual data acquisition device can also be mounted on or near therecorder's head. For example, the visual data acquisition device can beattached to a head harness that is worn by a recorder, as illustrated inFIG. 4, discussed above. The visual data acquisition device can also bemounted, for example, on a headband near the recorder's ear or on theside of a pair of eyeglasses worn by the recorder, as also discussedabove.

The audio data acquisition devices can be mounted, for example, using abracket which houses the audio data acquisition devices and can beattached to a strap so that the bracket can slide along the strap.

The adjustability of the mounting mechanisms for the audio and visualdata acquisition devices advantageously allow the positioning of theaudio and visual data acquisition devices to be changed to accommodatedifferent body sizes and shapes.

The data storage and system control unit 703 includes a data storagedevice for storing recording data. The data storage and system controlunit 703 also includes a system controller for controlling the operationof, and mediating interaction among, other devices of the recording unit700. In one particular embodiment of the recording unit 700, the datastorage and system control unit 703 can be embodied by a Sony VideoWalkman, GV-A500 Hi 8 video cassette recorder NTSC. The GV-A500 alsoincludes a 4 inch color LCD screen that can be used to display arecording. In another particular embodiment of the recording unit 700,the data storage and system control unit 703 can be embodied by a SonyGV-D300 mini-DV cassette recorder. The recording unit 700 also includesa power supply (e.g., battery); this can be included in the data storageand system control unit 703. When the above-indicated GV-A500 videocassette recorder is used to embody the data storage and system controlunit 703, one of the following Sony InfoLITHIUM™ batteries can be used:the NP-F550, the NP-F750 or the NP-F950, providing approximately 1.2,2.6 or 4.3 hours of recording time, respectively. The above-indicatedcassette recorders are devices that store recording data on a lineardata storage medium. Other devices that store recording data on anonlinear data storage medium, as discussed in more detail above, can beused.

The control interface unit 702 can be embodied generally in a manner asdescribed above. In particular, the control interface unit 702 can beembodied by the Sony RM-95 Remote Commander remote control unit. TheRM-95 can be connected to the GV-A500 video cassette recorder via theLANC port of that recorder. (The GV-A500 can also be controlled usingmechanisms that are part of the GV-A500.) It can also be desirable tosubstitute a coiled cord (e.g., as used with a telephone handset) forthe cord typically provided with the RM-95 to minimize dangling ofconnecting wires between the control interface unit 702 and the datastorage and system control unit 703.

Integration of the components of the recording unit 700 can readily beaccomplished by those of skill in the art in view of the descriptionherein. Communication between components of the recording unit 700 canbe implemented using wired or wireless techniques and apparatus, asknown to those skilled in the art. The use of wireless communicationbetween some or all of the components can be desirable to increaseflexibility in choosing mounting locations for the various components.If wired communication among components is used, the wires betweencomponents can be made of a length that enables a desired range ofrelative movement between the connected components as can be expected tooccur during use of a recording unit intended for use by a recorderengaged in vigorous physical activity.

FIGS. 8A-8C are perspective views of a recording unit 800 representing aparticular physical instantiation of the recording unit 700 of FIG. 7,the recording unit 800 illustrated in FIGS. 8A-8C as worn by a recorder850. As will be clearer from the description below, the recording unit800 can be made compact and lightweight. Additionally, the recordingunit 800 is constructed in a manner similar to a conventional backpack,providing a familiar look and feel to a recorder. The recording unit 800can also be constructed with relatively few wires connecting components,and so that those wires which are used are constrained to minimizeinterference with the recorder or other objects. The recording unit 800can also be implemented to have a relatively simple set of functionalcapabilities, thus reducing the amount of power required and enabling asmaller power supply to be used in the recording unit 800. The recordingunit 800 can also be made water resistant, which can be particularlyuseful if the recording unit 800 is used during activities such as waterskiing or snow skiing.

A harness 851 securely attaches the components of the recording unit 800to the recorder 850. The harness 851 can advantageously be implementedso that the components of the recording unit 800 can be detached fromthe harness 851 and used separately from the harness 851 or with anotherharness.

The harness 851 includes a waist strap 851 a, two shoulder straps 851 band 851 c, and a sternum strap 851 d. The harness 851 can be implementedso that one or more of the waist strap 851 a, shoulder straps 851 b and851 c, and the sternum strap 851 d can be adjusted (i.e., loosened ortightened about the recorder 850) to accommodate different body sizesand shapes (using, for example, conventional mechanisms used for thatpurpose with backpack straps) and to ensure that the harness 851 remainssecurely in place on the recorder 850 during vigorous physical activity.Additionally, the waist strap 851 a, shoulder straps 851 b and 851 c,and the sternum strap 851 d are arranged so as to distribute the weightof the recording unit 800 over the recorder's body.

The waist strap 851 a is preferably padded. It can also be desirablethat the waist strap 851 a be made wider at the back of the recorder850, particularly if the housing 852 and associated apparatus (describedbelow) are to be supported there, to facilitate distribution of theweight of the recording unit 800. In such case, the waist strap 851 acan be shaped to curve around the hip of the recorder 850.

The visual data acquisition device 853 is mounted on the shoulder strap851 c. As will be more readily apparent from the description below, thevisual data acquisition device 853 can also be mounted on the shoulderstrap 851 b. This flexibility in mounting can be desirable, for example,to accommodate left- and right-handed recorders.

The shoulder straps 851 b and 851 c, and sternum strap 851 d helpstabilize a visual data acquisition device 853 that is mounted on theshoulder strap 851 c. In particular, the shoulder straps 851 b and 851c, and sternum strap 851 d can be tightened to so that the harness 851(and, thus, the visual data acquisition device 853) is held tightly tothe body of the recorder 850.

Audio data acquisition devices 854 a and 854 b are mounted on shoulderstraps 851 b and 851 c, respectively. As can be appreciated from FIGS.8A and 8C, such mounting enables the audio data acquisition devices 854a and 854 b to be mounted symmetrically (or substantially symmetrically)on the recorder 850 (about a vertical plane parallel to the direction ofview from in front or back of the recorder 850). The audio dataacquisition devices 854 a and 854 b can also be mounted at the same (orapproximately the same) height on the recorder 850, if desired.

The device or devices that embody the data storage and system controlunit 703 (FIG. 7) are mounted within a housing 852. The housing 852 ismade of a material and constructed so that the device(s) inside thehousing 852 are protected from damage resulting from exposure tomoisture and debris, as well as damage from light impacts against thehousing 852. Tensioning straps 860 a and 860 b can be attached to thehousing 852 and the waist strap 851 a to help securely hold the housing852 on the recorder 850.

The housing 852 is attached to the waist strap 851 a of the harness 851so that the housing 852 can slide along the waist strap 851 a, thusenabling the data storage and system control unit 703 to be positionedat any location about the waist of the recorder 850. It is anticipatedthat the recorder 850 will typically choose to position the data storageand system control unit 703 at a location adjacent to the recorder'sback when the recorder 850 is participating in the event (see FIG. 8B),since it is expected that that position will typically minimizeinterference by the data storage and system control unit 703 withparticipation by the recorder 850 in the event, is a more comfortableposition for the recorder 850 to support the data storage and systemcontrol unit 703, and provides some protection from impacts for the datastorage and system control unit 703. Similarly, it is anticipated thatthe recorder 850 may want or need to interact with the data storage andsystem control unit 703 (e.g., view a display 857, as shown in FIG. 8C)while the recorder 850 is wearing the recording unit 800 and, further,that such interaction can most easily take place if the data storage andsystem control unit 703 is positioned in front of (or, at least, to theside of) the recorder 850 (see FIG. 8C). Attaching the data storage andsystem control unit 703 as described above allows the recorder 850 toeasily position the data storage and system control unit 703 in thesetwo positions as desired.

The housing 852 can alternatively be mounted on other parts of theharness 851 and/or the housing 852 can be mounted so that the housing852 remains in a fixed position rather than being able to slide along astrap of the harness 852.

A control interface device 855 is connected to electronics within thehousing 852. A pocket 856 is formed on the housing 852 so that thecontrol interface device 855 can be inserted into the pocket 856 whennot in use by the recorder 850. The pocket 856 can be formed with anopening on both the left and right so that the control interface device855 can be inserted into the pocket 856 from either side (this can bedesirable, for example, to accommodate left- and right-handedrecorders). Velcro™ can be used to close the openings after the controlinterface device 855 has been inserted.

The wiring (not numbered in FIGS. 8A-8C) used to connect the componentsof the recording unit 800, or conduits for guiding the wiring, can besewn into the harness 851, so as to minimize or eliminate entanglementof the wiring with the recorder or other object.

The harness 851 is desirably made of a material (e.g., Cordura™, adurable, water resistant material) that can be washed (e.g., using waterand appropriate cleaning agent). Further, the electronic components ofthe recording unit 800 are desirably constructed so that the electroniccomponents are sealed from moisture penetration and/or can be easilyremoved from the harness 851 to enable cleaning. For example, the visualdata acquisition device 853 can be readily detachable from the shoulderstrap 851 c and the housing 852 can be enclosed within a washablematerial (e.g. cordura).

The harness 851 can include one or more storage compartments or holders.For example, the harness 851 can includes a water bottle holder 858(FIG. 8B). The harness 851 can also include (e.g., constructed as partof the waist strap 851 a) one or more pouches, compartments and/orpockets for storage of a wallet or other valuables. It can beparticularly desirable that the harness 851 include one or more pouches,compartments and/or pockets that can be sealed to enable storage of(e.g., water, sun tan lotion, food) that may otherwise present danger ofdamaging other parts (especially electronic components) of the recordingunit 800. It can also be desirable that the harness 851 include one ormore pouches, compartments and/or pockets that provide a secureenclosure that prevent the objects (e.g., a wallet or other valuables)enclosed therein from falling out of the enclosure, thereby preventingthe objects from getting lost during the vigorous physical activity inwhich the recorder is participating. Further, the harness 851 can beimplemented to enable various device holders (such as are used withhandyman or policeman utility belts) to slide on to, for example, thewaist strap 851 a. This can advantageously eliminate the need for aseparate belt or harness for carrying such device holders. The harness851 can also be implemented to enable a hydration system (e.g., aCamelbak™ system) or water bladder to be attached thereto.

As illustrated in FIG. 8B, a battery compartment 859 extends out of thehousing 852. The battery compartment 859 can be opened to enable thebattery to be changed without opening the rest of the housing 852.Alternatively, the battery can be separated out as a separate componentand mounted at a different location on the recorder 850 (e.g., betweenthe shoulder blades) in order to further enhance the benefits (e.g.,distribution of weight) of separating the components of the recordingunit 800.

FIGS. 9A and 9B are perspective views of a recording unit 900representing another particular physical instantiation of the recordingunit 700 of FIG. 7. FIG. 9A is a front perspective view of the recordingunit 900 being worn by a recorder 950. FIG. 9B is a side perspectiveview when the recording unit 900 is not being worn by a recorder.

Generally, the recording unit 900 can be constructed similarly to therecording unit 800 described above with respect to FIGS. 8A-8C. Therecording unit 900 primarily differs from the recording unit 800 in thatthe construction of the harness (designated in FIGS. 9A and 9B by thenumeral 951) of the recording unit 900 is different than that of theharness of the recording unit 800. The harness 951 of the recording unit900 includes a waist strap 951 a and a shoulder-strap 951 b. The harness951 can be implemented so that one or both of the waist strap 951 a andthe shoulder strap 951 b can be adjusted (i.e., loosened or tightenedabout the recorder 950) to accommodate different body sizes and shapes,and to insure that the harness 951 remains securely in place on therecorder 950 during vigorous physical activity. The adjustability of theshoulder strap 951 b can be implemented using, for example, a releasablemechanism 955 a such as is commonly used for backpack shoulder straps.The adjustability of the waist strap 951 a can be implemented using, forexample, mating velcro strips 955 b and/or conventional backpack releasemechanisms. Since the harness 951 of the recording unit 900 includesfewer straps than the harness 851 of the recording unit 800, therecording unit 900 may provide the recorder 950 with a sense of freedomof movement that is greater than that sensed by the recorder 850. Arecorder may also perceive that the harness 951 is easier to put on andtake off than the harness 851.

As illustrated in FIG. 9A, a visual data acquisition device 953 can bemounted on the shoulder strap 951 b. Mounting of the visual dataacquisition device 953 on the shoulder strap 951 b can be implemented sothat the visual data acquisition device 953 can be moved to one ofvarious positions along the shoulder strap 951 b. (A mechanism foraccomplishing this is described in more detail in the above-referencedU.S. patent application Ser. No. 09/225,993.)

An audio data acquisition device 954 can also be mounted on the shoulderstrap 951 b. The audio data acquisition device 954 can be implemented sothat the audio data acquisition device 954 can be mounted at variouslocations on the harness 951 (it may be desirable, for reasons discussedelsewhere herein, to mount the audio data acquisition device 954proximate to the top of the shoulder of the recorder 950). Further, theaudio data acquisition device 954 can be implemented as multipleseparate devices, as discussed elsewhere herein. For example, the audiodata acquisition device 954 can be implemented as two microphones, eachof which can be clipped on (or otherwise attached) to the recorder 950or the harness 951. In such case, it can be desirable to mount themultiple devices symmetrically with respect to the body of the recorder950, as also discussed elsewhere herein.

As illustrated in FIG. 9A, the recording unit 900 includes a visualrecording display 957 that is attached to, or mounted on, the waiststrap 951 a. As illustrated in FIG. 9B, the recording unit 900 includesa pocket 952 within which one or more electronic components of therecording unit 900 can be positioned. Additionally, the harness 951 caninclude a pocket 956 which can be used for holding any of a variety ofobjects, as discussed elsewhere herein. The pocket 956 can beconstructed to include a zipper so that the pocket 956 can be opened toplace objects in the pocket 956 and closed to securely hold objectswithin the pocket 956.

The color of visible surfaces of components of a recording unitaccording to the invention such as the recording unit 700 can be chosento produce desired visual characteristics. For example, as discussedabove, it can be desirable that the appearance of a recording unitaccording to the invention be unobtrusive: making visible surfaces ofcomponents of the recording unit black can help accomplish that goal.Other colors can be chosen to connote the anticipated use of therecording unit, either generally (i.e., use during participation in avigorous activity) or particularly (i.e., use during participation in aparticular activity) or for other purposes. For example, yellow may beused for a recording unit intended for use in a sports activity, orcamouflage colors may be used for a recording unit intended for use inactivities such as hunting. Or, the colors of the recording unit can bechosen to complement the colors of clothing worn by the recorder (e.g.,match the color(s) of a uniform, integrate fashionably with therecorder's clothing).

Though not shown in FIG. 7 (or in FIGS. 8A-8C or FIGS. 9A and 9B), arecording unit according to the invention such as the recording unit 700can also include a marking mechanism having capabilities similar to thatdescribed above. In view of the anticipated uses of such a recordingunit, it is desirable to make the marking mechanism relatively simple.For example, the marking mechanism can be implemented to include asingle button that is used to effect control (e.g., specify mark(s)) ofthe marking mechanism. Additionally, it is anticipated that a recordermay often want to mark recording data obtained hours or days previously.Therefore, it can be advantageous to implement the marking mechanism sothat the marking mechanism is physically separated from the rest of therecording unit and so that the marking mechanism includes a non-volatiledata storage device that enables marking data to be stored on themarking mechanism, thus enabling the recorder to carry around therelatively small and simple marking mechanism without need to carry theentire recording unit. When the marking data is stored on the markingmechanism, the marking mechanism is constructed with a communicationsport that enables the marking data to transferred from the markingmechanism. Further, when the marking data is stored on the markingmechanism instead of with the recording data, temporal synchronizationof marks with recording data can be enabled in the manner describedbelow by including an LED and a real-time clock (RTC) as part of themarking mechanism. The marking mechanism can be constructed so that,during operation of the recording unit, the marking mechanism can beoperated (e.g., the button can be depressed for more than apredetermined duration of time) to cause the LED to flash, the markingmechanism being held in a position that enables the visual dataacquisition device to film the flashing LED. The above-describedsynchronization is performed each time that recording begins again afterhaving been stopped.

b. Hybrid Recording Unit

Above, embodiments of a recording unit according to the invention aredescribed in which the recording unit is constructed from componentparts that, together, provide the functionality of the recording unit.Recording units according to other embodiments of the invention can beconstructed by adding to a portable video recorder (e.g., camcorder,portable dockable videotape recorder (VTR)) one or more devices(referred to hereinafter as the “auxiliary device”) that providefunctionality of a recording unit according to the invention asdescribed above. (For convenience, such a recording unit according tothe invention is sometimes referred to herein as a “hybrid recordingunit.”) In particular, the auxiliary device can advantageously provideone or more of the following types of functionality described above: 1)marking capability; 2) position sensing (e.g., GPS position and timemeasurements, electronic compass measurements); 3) physiologicalmonitoring; and/or 4) biometric identification.

Some portable video recorders include an external communication portthat enables communication with an external device. For example, somecamcorders made by Sony Corporation (e.g., CCD-TR101) include such anexternal communication port that enables communication with an externaldevice in accordance with a communication protocol known as LANC (alsoreferred to as Control-L). Some camcorders sold by Ricoh (e.g., R-18H),Nikon and Canon (e.g., ES-900) also enable communication with anexternal device in accordance with the LANC protocol. Some camcordersmade by Panasonic (e.g., AG-196U) include an external communication portthat enables communication with an external device in accordance with adifferent, though comparable, communication protocol (referred to as a“5-pin editing terminal”). Some camcorders made by JVC (e.g.,GR-DVL9000) include an external communication port that enablescommunication with an external device in accordance with still anothercomparable communication protocol referred to as JLIP. Additionally, theIEEE 1394 (e.g., Firewire or iLink) communication protocol can be usedwith some camcorders (e.g., Sony DCR-PC1, Sony DCR DVR 900, PanasonicAG-EZ30U) to enable communication to an external device. Additionally,SMPTE time code input can be used with professional camcorders orportable dockable videotape recorders (VTRs), such as the Sony BVV-5dockable Beta-SP VTR, to enable synchronization with an external device.Such portable video recorders can be used to construct a hybridrecording unit according to the invention, since the communicationcapabilities of the portable video recorder enable the portable videorecorder and auxiliary device to coordinate operation as needed toprovide the additional functionality of the auxiliary device.

In addition to functionality such as marking capability, positionsensing, physiological monitoring and/or biometric identification, theauxiliary device includes a system controller that controls theoperation of other functional aspect(s) of the auxiliary device andmediates interaction between the auxiliary device and portable videorecorder, and mechanism(s) for communicating with the portable videorecorder (which communication can be wired or wireless). Theconstruction and operation of an auxiliary device of a hybrid recordingunit according to the invention can be implemented by those skilled inthe art in view of the description above of implementation of thefunctionality embodied in the auxiliary device, as well as thedescription below of other aspects of a hybrid recording unit accordingto the invention.

It may not be possible (or it may not be desirable) to store data(“auxiliary data”) acquired and/or produced by an auxiliary device onthe data storage medium (e.g., videotape, DVD or hard disk) used by aportable video recorder to store recording data. In particular, typicalconsumer camcorders (as opposed to camcorders used for professionalvisual recording work) do not enable auxiliary data to be stored on thedata storage medium used to store recording data. (All of theabove-indicated camcorders are consumer camcorders, except the PanasonicAG-196U and Panasonic AG-EZ30U.) Such camcorders may include anotherdata storage device on which auxiliary data can be stored, though thisis often not the case. If a portable video recorder does include anotherdata storage device, auxiliary data can be communicated to the portablevideo recorder via the external communication port and stored on thatdata storage device. If the portable video recorder does not includeanother data storage device, it is necessary to provide for storage ofauxiliary data on another data storage device. Typically, this can beaccomplished by including a non-volatile data storage device as part ofthe auxiliary device. Data storage devices that are relatively small andlightweight, and consume relatively little power are preferred, such as,for example, an EEPROM, flash memory, an Iomega Clik! drive, or an IBMMicrodrive. However, the auxiliary device can also be constructed tocommunicate (in particular, wirelessly communicate) auxiliary data to adata storage device that is part of neither the auxiliary device nor theportable video recorder.

If an auxiliary device of a hybrid recording unit according to theinvention stores auxiliary data on a data storage device separate fromthat on which recording data is stored by the portable video recorder,it is desirable to implement the hybrid recording unit with thecapability to establish a temporal correspondence between the auxiliarydata and recording data in order to minimize or eliminate the need forpost-event processing to temporally synchronize the auxiliary data andrecording data. (Herein, for convenience, a data storage medium on whichrecording data is stored is sometimes referred to as a “recording datastorage device,” while a data storage medium on which auxiliary data isstored is sometimes referred to as an “auxiliary data storage device.”)The above-described camcorder communication protocols can enable datarepresenting the value of a data counter of the camcorder (e.g., a tapecounter for a videotape) to be communicated to an external device. In ahybrid recording unit according to the invention, the value of the datacounter can be communicated to the auxiliary data storage device so thata temporal correspondence can be established between the auxiliary datastored thereon and the recording data stored on the recording datastorage device. As an event is being recorded, the current value of thedata counter can be periodically communicated to the auxiliary datastorage device and stored thereon in a manner that associates the datacounter value with the auxiliary data obtained at the same time.However, data counter values are typically not inherently associatedwith a particular storage location on a recording data storage device(i.e., any data counter value only indicates the location of theassociated recording data on the recording data storage device relativeto the location of other recording data acquired during the samerecording in accordance with the relationship to the data counter valueassociated with that other recording data), so that associating a datacounter value with particular auxiliary data does not by itself identifythe recording data obtained at the same time as that auxiliary data(i.e., does not establish a temporal correspondence between therecording data acquired during a recording and the auxiliary dataacquired and/or produced during the recording). Such a temporalcorrespondence can be established by additionally identifying thelocation on the recording data storage device at which recording dataassociated with any particular data counter value is stored (i.e, byestablishing a reference data counter value to which all other datacounter values can be related).

It is anticipated that portable video recorders with which an auxiliarydevice will be used to produce a hybrid recording unit according to theinvention will often make use of a linear (i.e., not random access) datastorage medium, such as a videotape. (For convenience, the followingdescription is given with respect to a portable video recorder thatstores recording data on a videotape; however, with appropriatemodification, as will be readily understood by those of skill in theart, the description is generally applicable to portable video recordersthat make use of any linear data storage medium.) Further, many of suchportable video recorders do not write a time code to the linear datastorage medium (Hi-8 with RC time code and mini-DV, for example, do)indicating the time at which corresponding recording data was obtained.In general, at the beginning of use of such a portable video recorder,the videotape can be wound to any position (the “initial recordingposition”), i.e., recording can begin at any location on the videotape.For portable video recorders that don't store time code with therecording data, to establish a temporal correspondence between theauxiliary data and the recording data, it is necessary to identify theinitial recording position and communicate that information to theauxiliary data storage device. As indicated above, a communicationprotocol (e.g., Sony's LANC) that enables tape counter data to becommunicated to the auxiliary data storage device can be used for thispurpose. Prior to beginning recording, the value of the tape counter atthe initial recording position is identified. The videotape is thenrewound to the beginning (a “beginning data storage position”) of thevideotape. The value of the tape counter at the beginning data storageposition of the videotape is identified. The two tape counter values arecompared to each other (e.g., subtracted) to establish a tape counteroffset that represents the number of tape counter counts from thebeginning data storage position of the videotape to the location of theinitial recording position on the videotape. The videotape is wound tothe initial recording position so that recording can begin. (The tapecounter values and notification that the videotape is at the beginningdata storage position or the initial recording position can beascertained by the auxiliary device and/or the auxiliary data storagedevice via communication with the portable video recorder in accordancewith the communication protocol.) The beginning of recording can definea tape counter value of zero for the auxiliary data (which alsocorresponds to the location on the videotape defined by the tape counteroffset). During recording, the portable video recorder can communicateincremental tape counter values (relative to the tape counter value atthe beginning of recording) to the auxiliary data storage device, whichstores each incremental tape counter value so that the value isassociated with auxiliary data obtained at that time. Each incrementaltape counter value defines the location on the recording data storagedevice, relative to the location defined by the tape counter offset, atwhich recording data is stored that corresponds to the auxiliary dataassociated with the incremental tape counter value. Thus, as can beseen, the determination of the tape counter offset enables a temporalcorrespondence to be established between the recording data and theauxiliary data.

There may be no need to calculate a tape counter offset (or other,comparable data counter offset) in some hybrid recording units accordingto the invention (e.g., when the hybrid recording unit includes acamcorder that uses the LANC protocol and writes RC time code to thevideotape, or when the hybrid recording unit includes a mini-DVcamcorder). However, in such hybrid recording units according to theinvention, it can be desirable to ensure that recording begins at alocation on the portable video recorder videotape immediately followingthe location at which the most recently acquired recording data isstored. (A current location at which recording data is to be stored maynot be the same as such location due to, for example, rewinding of thevideotape to enable display of previously stored recording data.) In ahybrid recording unit including a mini-DV camcorder (which stores a timecode on the videotape as each video frame is acquired, and associatesthe time code with that video frame), for example, to ensure that newlyacquired recording data is stored beginning just after the location onthe videotape of the most recently acquired recording data, the hybridrecording unit can be implemented to operate in accordance with thefollowing procedure. When recording is to begin, a determination is madeas to whether a time code is stored at the current location of thevideotape, in a manner as known to those skilled in the art. If not,then the videotape is rewound to a location at which storage of a timecode on the videotape is detected, also in a manner as known to thoseskilled in the art, or until the beginning of the videotape is reached.In either case, recording begins at the location to which the videotapeis rewound. If, on the other hand, a time code is stored at the currentlocation, then the videotape is wound forward until a section of thevideotape is reached at which no time codes have been stored. Then, thevideotape is rewound as described above, recording beginning at thelocation to which the videotape is rewound. The above-describedoperation of a hybrid recording unit according to the invention caneither be automatic (i.e., can occur each time recording begins, withoutexplicit instruction from a recorder) or manual (i.e., can occur uponprovision to the hybrid recording unit of an appropriate instruction orinstructions by the recorder).

The auxiliary device can be implemented so that determination of a tapecounter offset (or other, comparable data counter offset) as describedabove can occur automatically upon insertion of a videotape into theportable video recorder. Upon determining that a videotape has beeninserted into the portable video recorder (this can be ascertained viacommunication with the portable video recorder in accordance with thecommunication protocol), the auxiliary device identifies the currenttape counter value, then transmits an instruction to the portable videorecorder via the external communication port to cause the portable videorecorder to rewind the videotape. (Some portable video recorders mayonly enable rewinding of the videotape when the portable video recorderis operating in a particular mode, e.g., VCR mode; for portable videorecorders for which this is true, a recorder will have to establish thatmode of operation of the portable video recorder—if not alreadyestablished—before the videotape can be rewound.) When the auxiliarydevice identifies that the videotape has been rewound to the beginningof the videotape, the auxiliary device identifies the current tapecounter value. The auxiliary device then transmits an instruction to theportable video recorder to cause the portable video recorder to wind thevideotape until the tape counter value of the initial recording positionis reached. Before, during and/or after the videotape is wound back tothe initial recording position, the auxiliary device compares the twotape counter values to determine the tape counter offset.

If it is known that the videotape is ready for recording at thebeginning of the videotape, it is not necessary to determine a tapecounter offset (or other, comparable data counter offset) as describedabove. Rather, the tape counter value can be identified by the auxiliarydevice and recording can begin immediately. While the method fordetermining a tape counter offset as described above can be used in sucha circumstance (the tape counter offset will be determined to be zero),some time will elapse (e.g., several seconds) while the auxiliary deviceand portable video recorder operate to make the tape counter offsetdetermination, such that content of interest is not recorded while thetape counter offset determination is made. It can be preferable toimplement the auxiliary device to provide a control mechanism thatenables a recorder to indicate that no tape counter offset determinationis necessary, so that recording can begin immediately or nearlyimmediately (in any event, appreciably sooner than would otherwiseoccur). More generally, a recorder may find it desirable to use suchcontrol mechanism at any time, not only when it is known that thevideotape is, ready for recording at the beginning of the videotape (atemporal correspondence between the auxiliary data and recording datacan be established as part of post-event processing of the auxiliarydata and recording data).

A temporal correspondence between the auxiliary data and recording datacan be established in other ways. For example, a predetermined visualphenomenon (e.g., a flash of light) or aural phenomenon (e.g., ahandclap or emission of a predetermined tone) that will be recorded bythe portable video recorder can be generated at the beginning ofrecording. The beginning of the auxiliary data corresponds to theoccurrence of the visual or aural phenomenon in the recording datastored on the recording data storage device.

II. Post-Event Use of Event Recording

A. Processing of Recording Data

1. In General

As discussed above, the recording data stored by a recording unit istypically digital data, either because of conversion of analog recordingdata acquired by the recording unit or because digital data acquisitiondevices were used to acquire the recording data. However, even if therecording data stored by a recording unit is analog, it is desirable toconvert the analog recording data to digital form to enable a widerrange of processing and display of the recording data as describedbelow.

It may be desirable to temporally synchronize multiple recordings of thesame event that were simultaneously obtained by different recordingunits according to the invention. Such synchronization can be desirableif it is desired to display the recordings in a manner that enablessimultaneous observation of the recordings. Further, suchsynchronization is necessary if the content of multiple recordings is tobe integrated in some manner to produce a display, as described furtherbelow in the discussion of ways of displaying recordings. As describedabove, a recording unit according to the invention typically includes aclock that is used to time-stamp recording data as the recording data isacquired. However, the clocks of different recording units may not beadequately synchronized to enable use of the time-stamps associated withthe recordings obtained from different recording units to synchronizethose recordings. (If recording units according to the invention eachinclude a GPS receiver, such mis-synchronization will only exist to anappreciable degree if one or more recording units are shielded from aGPS transmitter system for an extended period of time, e.g., severalhours.) Or, the recording units may not include clocks for enablingtime-stamping of the recordings.

The content of multiple recordings can also be used to establish asynchronization between recordings (or to double-check a synchronizationbased upon time-stamps). In general, either the content of visualrecordings or the content of audio recordings (or both) can be used toestablish synchronization between the recording data obtained bydifferent recording units.

For example, when two different visual recordings capture (at least inpart) the same content, that overlapping visual content can be used tosynchronize the visual recordings. During display of the visualrecordings, one of the visual recordings can be paused while content isbeing displayed that is known to also be present in the other visualrecording. The display of the other visual recording can then be movedeither forward or backward in time until the same content is displayedin that visual recording. The locations within each visual recording canthen be marked as occurring at the same time, thus synchronizing thevisual recordings.

Because pausing an audio recording allows no sound to be displayed,synchronizing audio recordings cannot be done as easily as synchronizingvisual recordings as described above and may not be sufficientlyaccurate. An analogous synchronization of audio recordings would requiresuccessive starting and stopping of the audio recordings to zero in onthe overlapping audio content. Thus, when visual recordings includeoverlapping content, it is typically preferable to use the visualrecordings to synchronize the recording data. Nevertheless, due to the(typically) relatively limited field of view of the visual recordings,it may be the case that there is little or no overlapping content in thevisual recordings from different recording units, thus making the use ofvisual recordings to establish a synchronization difficult orimpossible. (However, anticipating the need for later synchronization,recorders can intentionally operate their recording units for a shortperiod of time to acquire overlapping visual content.) The audiorecordings can be used to establish a synchronization in such case,since audio recordings typically include content from all around therecorder, thus increasing the likelihood that there will be overlappingcontent in the audio recordings obtained by different recording units.

It is anticipated that many users of a recording unit according to theinvention will not have access to sophisticated methods and apparatusfor processing recording data after an event. Advantageously, the abovemethods of synchronizing the post-event display of multiple recordingscan be implemented without use of sophisticated methods or apparatus.However, synchronization of multiple recordings can also be accomplishedusing other (perhaps more sophisticated) methods and/or apparatus. Forexample, the audio and/or visual recording data from multiple recordingscan be analyzed and compared to determine the time of occurrence of thesame content within different recordings, thereby enabling therecordings to be synchronized.

As discussed above, a recording unit according to the invention can beadapted to enable marking of the recording as the recording is obtained,each mark specifying a time or duration of time during the recording.The capability can be provided to specify new marks (including definingnew types of marks) as part of post-event processing of the recordingdata. The capability to modify existing marks in the recording data canalso be provided, e.g., move the location (time) of a mark within therecording data, change the type of mark specified at a particularlocation within the recording data. The temporal vicinity of a mark(e.g., the exact time in the recording specified by the mark, a time inthe recording that is a predetermined amount of time before or after thetime specified by the mark, or a specified duration of time relative tothe time specified by the mark) can also be specified as part ofpost-event processing of the recording data.

Often, it will be necessary to process the recording data to enabledisplay of one or more recordings in a particular manner. For example,as indicated above, it may be desirable to integrate the content frommultiple recordings to produce a display. Recording displays in whichcontent from multiple recordings is integrated are discussed furtherbelow, together with the processing of recording data necessary ordesirable to effect such displays. Other ways of displaying a recording(e.g., summarization of a recording, variation in the speed of displayof a recording) may also require processing of the recording data in aparticular manner; such processing is described below, as appropriate,with the discussion of such

2. Visual Data Processing

It may be necessary or desirable to process the visual recording data toameliorate undesirable characteristics of the visual recording. Forexample, since a guiding principle of the invention is that therecording of an event be obtained in a manner that is, in some sense,incidental (i.e., in a manner that requires little interaction by therecorder), it will often be the case that the visual recording suffersfrom “shake” or “jerk” (i.e., the field of view of the visual recordingrapidly and repetitively goes through wide ranges of movement) thatresults from motion of the recorder during the recording (e.g., arhythmic motion of the visual data acquisition device—with shocks atextremes of the motion—as the recorder walks or runs; or an abruptmovement of the visual data acquisition device as the recorder jumpsfrom a ledge, trips over a curb or rapidly turns to look in anotherdirection). Additionally, as discussed above, as the lens angle (i.e.,field of view) of the visual data acquisition device increases, thedistortion (fish-eye effect) at the edges of the recorded image canincrease. It will often be desirable to remove such artifacts from thevisual recording. However, in each case, whether to incur thecomputational cost of processing the visual recording data to improvethe quality of the visual recording should be decided in view of themagnitude of the improvement: some improvements may be insufficientlygreat to warrant the cost associated with processing the recording data.

In general, any of a variety of known methods can be used to processvisual recording data to reduce or eliminate the above-describedundesirable characteristics of a visual recording. Such methods aredescribed in, for example, the following papers, the disclosures ofwhich are incorporated by reference herein: 1) “Creating Full ViewPanoramic Image Mosaics and Environment Maps,” by Richard Szeliski etal., Computer Graphics Proceedings, Annual Conference Series, 1997, pp.251-258 (referred to hereinafter as the “first Szeliski publication”);2) “Video Mosaics for Virtual Environments,” by Richard Szeliski, IEEEComputer Graphics and Applications, March 1996, pp. 22-30 (referred tohereinafter as the “second Szeliski publication”); 3) “Virtual Bellows:Constructing High Quality Stills from Video,” by S. Mann et al., FirstIEEE International Conference on Image Processing (ICIP-94), Vol. 1,November 1994, pp. 363-367 (referred to hereinafter as the “first Mannpublication”); and 4) “Video Orbits of the Projective Group: A simpleapproach to featureless estimation of parameters,” by S. Mann et al.,IEEE Transactions on Image Processing, September 1997, Vol. 6, No. 9,Oct. 15, 1996, pp. 1-31 (referred to hereinafter as the “second Mannpublication”). Additionally, a technique developed by NASA and known asVideo Image Stabilization and Registration (VISAR) can be used to reduceor eliminate blurring and other artifacts in a visual image.

For example, when the field of view of the visual recording undergoesrapid, repetitive movements (e.g., movements in rhythm with therecorder's walking or running), there will typically be content,generally located centrally within the range of visual content duringsuch movement, that remains continually in the visual recording. Theappearance of movement in the visual recording display can be reduced oreliminated by analyzing the visual recording data (e.g., in a videorecording, comparing each successive frame of video data to one or moreof the immediately previous video frames) and discarding visualrecording data (for convenience, referred to herein as “transient visualrecording data”) that represents content that does not remain“continually” in the visual recording. Any appropriate method can beused to determine which visual recording data is transient (and shouldnot be used to generate the visual recording display) and which is not(see, e.g., the first and second Mann publications referenced above). Inparticular, the analysis of the visual recording data may incorporateconsideration of the frequency with which content moves into and out ofthe visual recording: a high frequency indicates rapid movement that maymake elimination of transient visual recording data desirable, while alow frequency indicates slow movement that may make elimination oftransient visual recording data unnecessary and undesirable.Alternatively or additionally, the visual data acquisition device can beinstrumented (with, for example, one or more accelerometers) to identifymovement of the visual data acquisition device that results in movementof the field of the view of the visual data acquisition device, and thatinformation used to identify whether—and, if so, which—visual recordingdata should be eliminated. The display window can be varied in size inaccordance with the amount of visual content that is discarded.

Another approach to processing visual recording data to amelioraterapid, repetitive movement of the field of view of the visual recordingis to move the display window in synchronization with the movement sothat distraction associated with movement of the field of view of thevisual recording is minimized or eliminated. As before, transient visualrecording data is identified and eliminated. Content (for convenience,referred to herein as “focal content”) that, does not correspond totransient visual recording data is displayed at a fixed location on thedisplay screen of a display device. Content that moves into and out ofthe visual recording is added to and removed from, respectively, thedisplay of the visual recording at a location on the display screen thatis appropriate for that content. Thus, while the display of the visualrecording appears to move around on the display screen, the focalcontent remains displayed at a fixed location (i.e., does not appear tomove).

Still another approach to processing visual recording data to amelioraterapid, repetitive movements of the field of view of the visual recordinginvolves using a visual data acquisition device having a field of viewlarger than the area of the display window within which the visualrecording will be presented. Since the visual data acquisition devicefield of view is larger than the area of the display window, some of thevisual recording data is eliminated prior to generating the visualrecording display. Identification of transient visual recording data canbe one consideration in determining which visual recording data is to beeliminated.

Yet another approach to processing visual recording data to amelioraterapid, repetitive movements of the field of view of the visual recordingis to display only some of the frames of visual recording data.Displaying single frames for longer periods of time can create an imagethat appears to be more stable than would be the case if all of theframes of a sequence of frames of visual recording data were displayed.

As the lens angle of a visual data acquisition device increases, linesat the edges of the recorded image start to appear curved. This“fish-eye” effect can be corrected using techniques known to thoseskilled in the art (see, e.g., the second Mann publication referencedabove, U.S. Pat. No. 5,185,667 and U.S. Pat. No. 5,796,426).

Additionally, when successive images of a visual recording include thesame content in part or in whole, the parts of those images containingthat content can be analyzed and combined to produce an enhancedrepresentation of that content (see, e.g., the second Mann publicationreferenced above, as well as the NASA VISAR technique, also referencedabove). Even when there are no distortions in a visual recording, thisprocessing can be used to improve the quality of the visual recording.

Above, ways of reducing or eliminating distortion of a visual recording,or enhancing the quality of a visual recording, have been discussed. Itmay sometimes be desirable to introduce distortion into a visualrecording to produce particular visual effects.

For example, the visual data can be processed, using techniques known tothose skilled in the art (see, e.g., the first and second-Mannpublications referenced above), to produce a visual recording display inwhich images are stretched or squeezed, or in which the aspect ratio ofthe image is changed. Stretching the visual recording display can bedesirable to enable more visual content to be displayed. Squeezing thevisual image can be desirable to enable information other than thevisual recording to be displayed, such as, for example, informationconcerning the content being displayed, the time of acquisition of thedisplayed content, and the location at which the displayed content wasacquired. Changing the aspect ratio (i.e., stretching and/or squeezingthe height and width of the visual recording display in different ways)can be desirable to accommodate one or both of displaying more visualcontent and displaying other information.

As another example, the displayed image can be rotated with respect tothe orientation at which the image was acquired. As an illustration, ifthe recorder was bent over or otherwise positioned so that, when visualrecording data was being acquired, the visual data acquisition devicewas rotated with respect to the normal orientation of the visual dataacquisition device, it may be desirable, when displaying the visualrecording, to rotate the displayed image in an amount equal and inopposite direction to the rotation of the visual data acquisitiondevice. Thus, a visual recording display can be produced in which aconstant orientation of the displayed image is maintained regardless ofrotation of the visual data acquisition device during acquisition of thevisual recording data.

Other types of distortions that can be introduced into a visualrecording display can actually help smooth a visual recording displaythat would otherwise appear jerky. Processing of video recording data toachieve this effect is known to those skilled in the art.

3. Audio Data Processing

In general, any appropriate method for processing audio data to reduceor eliminate distortion can be used as necessary to process audiorecording data obtained by a recording unit according to the invention.For example, when multiple audio recordings are displayed together(either the recordings obtained by multiple audio data acquisitiondevices of a single recording unit or recordings obtained by multipleaudio data acquisition devices of different recording units), thedifferences in position of the audio data acquisition devices and/ordifferences in synchronization of the audio recordings may produce echo.However, as known to those skilled in the art, methods exist forprocessing multiple audio recordings to reduce or eliminate echo; thosemethods can be used for the same purpose in processing audio recordingsobtained by recording units according to the invention. Further, theposition information and time codes that can be obtained by a recordingunit according to the invention can be used to identify the position ofeach recording unit at any particular time, thus facilitating suchprocessing by enabling the delays between recording units to becalculated.

Additionally, like visual recording data obtained by a recording unitaccording to the invention, it may be desirable to process audiorecording data obtained by a recording unit according to the inventionto add distortion to the audio data. For example, it may be desirable toprocess the audio recording data to eliminate or reduce background noiseor identifiable sounds that are undesirably present in the audiorecording. Conventional techniques, known to those skilled in the art,can be used to process audio recording data in this way.

Processing of the audio recording data can be performed for otherpurposes as well. As discussed above, beam forming can be used todiscriminate among various sound sources in an audio recording. In thediscussion above, beam forming was used to eliminate audio data duringthe recording of an event so that only audio data that was anticipatedto be of particular interest or importance to the recorder is stored ona data storage device of the recording unit. Beam forming can also beused after an event as part of the post-event processing of the audiorecording. This may be preferable to performing beam forming duringrecording of the event, since, as discussed above, the latter may resultin the loss of valuable audio recording data. Further, particularly aspart of post-event processing of recording data, beam forming can beperformed using audio recording data obtained by microphones frommultiple recording units.

The implementation of beam forming is known by those skilled in the artand is described in more detail in, for example, “A microphone arrayfor,car environments,” by Yves Grenier, Speech Communication, Vol. 12,No. 1, March 1993; “Computer-steered microphone arrays for soundtransduction in large rooms,” by J. L. Flanagan et al., J. Acoust. Soc.Am., Vol. 78, No. 5, November 1985; and “An Algorithm for LinearlyConstrained Adaptive Array Processing,” by Otis Lamont Frost III,Proceedings of the IEEE, Vol. 60, No. 8, August 1972, the disclosures ofwhich are incorporated by reference herein. Beam forming can be used,for example, to identify the location from which sounds emanate. Asdiscussed above, it may be desirable to eliminate audio recording datathat corresponds to sounds which do not emanate from a location in frontof the recorder. Beam forming can also be used to enable the display ofsounds only from particular sources. For example, it might be desired todisplay only the speech of a particular person or group of persons. Beamforming can be used to identify the audio recording data that includesthe speech of that person or group of persons. Only that audio recordingdata is then used to generate a display of the audio recording. The useof beam forming to identify audio data corresponding to particularspeakers is described in, for example, U.S. Pat. No. 5,694,474, entitled“Adaptive Filter for Signal Processing and Method Therefor,” issued toJohn-Thomas Calderon Ngo et al., on Dec. 2, 1997, the disclosure ofwhich is incorporated by reference herein.

The audio recording data can also be augmented with other audio dataduring post-event data processing. For example, music can be added tothe audio recording data.

B. Display of Recorded Event

1. In General

It is anticipated that, frequently, multiple recording units accordingto the invention will be used to simultaneously obtain multiplerecordings of an event. As described further below, it can be desirableto simultaneously display multiple recordings of an event that weresimultaneously obtained by different recording units. However, thecapability of viewing any single recording of an event, selected frommultiple recordings of the event, and the capability of serially viewingmultiple recordings of an event, are also very desirable in their ownright.

In particular, since a recording unit according to the invention isintended to be operated with little or no attention from the recorder,it can be expected that, at many times during an event, the recordingobtained by a particular recording unit will be deficient in some way.For example, content of interest may not be captured in the recording.Or, content of interest may not be captured as well as desired (e.g.,visual content is not framed as desired or background noise obscuresaudio content of interest). When multiple recordings of an event areobtained, there is a reasonable likelihood that, at a time when therecording obtained by a particular recording unit is deficient in someway, another recording unit obtained a recording that does not sufferfrom that deficiency. (Or, if neither recording is deficient, onerecording may still be better in some way.) For example, as a result ofdifferences in proximity to the source of audio content of interest, onerecording may have captured the audio content better than anotherrecording and/or may have acquired visual recording data have betterresolution than the other recording. Or, if the recording obtained byone recording unit has background noise that obscures a conversation ofinterest, a recording obtained by another recording unit may well haveobtained a recording of that conversation that is not so obscured. Or,if one recording unit missed filming particular activity of interest,the existence of other recordings provides the possibility that theactivity was captured in one of those recordings. In general, then, theexistence of multiple recordings tends to increase the quality of therecording of an event.

Additionally, each recorder, whose recording unit will not record visualrecording data including the recorder's participation in an event(unless the recorder allows another participant to use the recordingunit or, at least, the visual data acquisition device of the recordingunit), can be expected to be desirous of viewing, in addition to therecording obtained by the recorder's recording unit, a recording of theevent that includes visual content in which the recorder is present. Theexistence of multiple recordings increases the likelihood that, at anygiven time, a recorder is present in a visual recording of the event.

Rather than serially displaying entire recordings of an event, it can bedesirable to serially display segments of multiple recordings that aretemporally coincident (i.e., to view a particular activity as recordedby different recording units). It may further be desirable to displaythe same audio recording with each visual recording segment, either byselecting one of the audio recordings or by blending the multiple audiorecordings (as discussed further elsewhere herein).

As indicated above, it is also desirable to simultaneously displaymultiple different recordings of the same event. Like the serial displayof single recordings, displaying multiple recordings simultaneously canenable each recorder to observe (view or listen to) activity during anevent of which the recorder may not have been aware, as well as viewcontent that includes the recorder (which typically will not be part ofthe recording obtained by that recorder). Further, it will usually bedesirable that multiple recordings displayed at the same time betemporally synchronized. (An initial synchronization can be accomplishedas described above.) The concurrent viewing of synchronized recordingsobtained from different viewpoints during an event is an informative andinteresting viewing experience that cannot be obtained by a viewing asingle recording from one point of view.

Some display devices may not adequately maintain synchronization of thedisplay over time. Typical consumer videocassette recorders (VCRs) aresubject to playback drift that over a relatively short time will resultin an unacceptable loss of synchronization between recordings displayedby different VCRs. Expensive VCRs (e.g., those currently costing about$5000 to $40,000, such as the Sony EVO-9850 Hi-8 VCR and the PanasonicA6-DS555 S-VHS VCR) operating under control of an edit controller thatis connected to, or includes, a common time-based correcting unit do notsuffer from this problem.

It can further be desirable to integrate the content of multiplerecordings in some manner. For example, the displays of multiplerecordings can be located with respect to each other (either on a singledisplay device or multiple display devices) in a manner that replicatesthe spatial relationship of the content of the respective recordingswithin the environment in which the recordings were obtained. Usingtechniques known to those skilled in the art, as discussed furtherelsewhere herein, position information (and, perhaps, the relatedderivative information) obtained by position sensing devices of therecording units that obtained the recordings can be used to enable suchspatial integration of the displays of multiple recordings, since theposition information can be used to determine the relative locations andorientations of those recording units throughout the recordings. It mayalso be possible to use the content of the recordings themselves tospatially integrate the recording displays (especially if the content ofthe recordings overlaps), again using techniques known to those skilledin the art.

Multiple recordings can be blended to form a single displayed recording.Ways of blending visual recording data and audio recording data arediscussed in more detail below.

Multiple recordings can also be integrated in other ways. For example, asingle integrated recording display can be formed by combining segmentsof different recordings obtained at different times. Such a recordingmay further be characterized in that the successive recording segmentsare temporally synchronized so that the end of one segment correspondstemporally with the beginning of the next segment. Thus, for example, asingle display of the entire event can be constructed in which differentrecordings are being displayed at various times. Such a display may bedesirable, for example, as a way to concisely display the “best” overallcontent obtained by all of the recording units during the event.

A single integrated recording display can also be formed by combiningdifferent types of recording data obtained at the same time by differentrecording units. For example, the audio recording obtained over aparticular duration of time by one recording unit can be combined withthe visual recording obtained over the same duration of time by adifferent recording unit. Again, such a display may be desirable as away to concisely display the “best” overall content obtained by all ofthe recording units during the event. Similarly, such a display may becreated to display the best audio recording for a particular visualrecording or vice versa. As an illustration, one recorder may obtain avisual recording of a conversation between two other recorders: it maybe preferable to use that recorder's visual recording of theconversation because neither of the visual recordings obtained by theother two recorders will include both conversing recorders. However, thebest recording of the speech of each recorder will typically be theaudio recording obtained by that recorder. Thus, it may be preferable toalternately use the audio recordings of those two recorders, dependingupon who is speaking at any given time. Or, the audio recordingsobtained by those two recorders can be blended (as described elsewhereherein) and displayed as a single audio recording.

A single integrated recording display can also be formed by locating andcombining different views of the same space. The views can be taken fromdifferent recordings or a single recording.

Multiple recordings can also be displayed simultaneously in anon-integrated way. Examples of such visual and audio recording displaysare discussed in more detail below.

As described above, a recording unit according to the invention can beadapted to enable marking of a recording as the recording is obtained.(As indicated above, each mark can be used to specify a time or durationof time during the recording and can have associated therewith apredetermined meaning.) Marks can be used to search, display and/orprocess recording data. Such searching, display and/or processing cantake place during recording of an event (if the recording unit isadapted to enable such functionality) or after recording of an event(including after recording and marking data has been transferred toanother device). To make maximum use of the flexibility in searching,displaying and/or processing a recording that is provided by marking therecording, the recording data is preferably stored on a non-linear datastorage medium (such as a conventional hard disk or DVD disk) thatenables immediate access to any of the recording data, rather than on alinear data storage medium, such as a tape.

The marks can be used to effect display of the marked recording in aparticular manner. In general, a mark enables an observer of a displayedrecording to quickly move to recording content in the temporal vicinityof the marked part of the recording. (What can constitute “temporalvicinity” is discussed in more detail above.) For example, marks can beused as a tool for enabling skimming of a recording, replay of a part orparts of a recording, or summarizing of a recording. Marks can be usedto control display of a recording either automatically or under controlof an observer of the recording. For example, all marked parts of arecording, or all parts of a recording marked by a mark having aparticular meaning, can be displayed automatically. (Automatic displayfurther necessitates that both the beginning and end of a marked part beidentified in some manner, either explicitly or relative to a specifiedlocation in the recording, e.g., the location of a mark.) Similarly,during observation of a recording, an observer could (with appropriateadaptation of the display device, as can be understood by those skilledin the art) control a display device to fast-forward or skip to a partof a recording marked by any mark or a mark having particular meaning.

As discussed above with respect to non-contemporaneous marking (i.e.,retrospective and predictive marking), in addition to specifying thetime of a mark, a confidence level can be specified indicating the levelof certainty that the time of the mark corresponds to the content thatthe recorder is seeking to mark. A specified confidence level can beused to affect the display of a corresponding marked part of therecording. For example, the magnitude of the confidence level can beused to determine a duration of time prior to the marked time at whichto begin the display: as the confidence level decreases, the duration oftime increases, thus increasing the likelihood of displaying the contentof interest. Or, for example, if the mark specifies a particular timeduring the recording, the magnitude of the confidence level can be usedto determine a duration of time, relative to the marked time, todisplay: as the confidence level increases, the duration of timedecreases. Illustratively, a 50% confidence level might cause thedisplay of a minute of a recording (centered about the time of themark), a 75% confidence level might cause the display of 45 seconds ofthe recording, and a 100% confidence level might cause the display of 30seconds of the recording. The durations of display can also be adjustedin accordance with the recency of a mark, e.g., as the duration of timebetween the marked time and the marking time increases, the duration ofthe display can increase a corresponding amount. The magnitude of theconfidence level can also be used to affect the speed with which arecording is displayed. For example, the recording display speed candecrease as the confidence level increases. Further, the manner in whichthe recording is displayed based on the confidence level (as well as themagnitude of the range of times and/or the confidence level, asdiscussed above) can be made dependent upon the recency of the mark.Additionally, the recording unit can be implemented so that a confidencelevel is automatically associated with a mark based on the recency ofthe mark. For example, the confidence level can decrease with increasein the duration of time between the marked time and the marking time.

The use of marks to effect display of a recording entails use of themarks to search the recording data. Marks can also be used to enablesearching of recording data without subsequent display. A recording canbe searched to identify any marked part of a recording or only parts ofa recording marked with a particular type of mark or marks. For example,it may be desirable to search recording data to identify particularmarked parts of a recording and discard parts of the recording data thatwere not recorded within a predetermined duration of time from the timeof the marks.

Though it is contemplated that it will be particularly useful for arecorder to be able to specify marks in a recording as the recording isbeing obtained, it can also be useful to provide a device that enables arecorder or other person (e.g., an observer of the recording displayother than a recorder) to add marks to the recording (including definingnew types of marks), modify existing marks (e.g., move the location of amark within the recording data, change the type of mark specified at aparticular location within the recording data), or delete existing marksafter the recording has been obtained (i.e., post-event). The temporalvicinity of a mark (e.g., the exact time in the recording at which themark was specified, a time in the recording that is a predeterminedamount of time before or after the time at which the mark was specified,or a predetermined duration of time relative to the time at which themark was specified) can also be specified as part of post-eventprocessing of the recording data. Nevertheless, it may be desirable toimplement such capability so that it can be disabled by, for example,the recorder who obtained the recording. It may also be desirable toenable one or more people that add marks to prevent (eitherautomatically or at the direction of the person adding the mark)deletion or modification of the marks that they add.

As indicated above, a mark enables an observer of a displayed recordingto quickly move to recording content in the temporal vicinity of themarked part of the recording. (What can constitute “temporal vicinity”is discussed in more detail above.) The display device can be adapted toallow the observer to specify the temporal vicinity of a mark as theobserver is observing a recording.

The display of a recording can be augmented with additional informationregarding data acquired during recording in addition to recording data,as well as information derived therefrom. In particular, suchinformation can concern the “recording state” at the time displayedcontent was recorded. For example, the time of recording, positioninformation (i.e., location of the recorder and/or orientation of thedata acquisition device) obtained by a position sensing device, relatedderivative information (e.g., velocity of the recorder) determined fromthe position information, information regarding control settings of thedata acquisition device (e.g., setting of the lens focusing mechanism,white balance, pedestal, aperture and/or gain of the visual dataacquisition device, or setting of the audio pickup pattern of the audiodata acquisition device), or physiological monitoring information(either raw physiological data, or physiological monitoring informationgleaned from raw physiological data, such as the level of excitement ofthe recorder) can be displayed. It is also possible, after reviewing adisplay of the recording, to add information to the display regardingthe content of the recording, such as a description of the location atwhich the recording occurred, the activity recorded, and a person orpersons that were recorded. Other types of information, such as therecorder's thoughts at the time particular content was recorded orthoughts about the content that was recorded can also be added to thedisplay. Additionally, some of the above-described information can bedisplayed in context. For example, an indication of the location of therecorder at the time when currently displayed content was recorded canbe displayed within a map of the space within which the recorded eventtook place. Or, for example, the time at which particular content wasobtained can be displayed within a representation of the duration of therecording, indicating, for example, whether the recorded activity tookplace near the beginning, middle or end of the event. Augmenting adisplay with additional information as above can be particularlydesirable when the recording content remains substantially constant overa period of time. For example, an audiovisual recording of a ski run canbe enhanced by a display of the skier's level of excitement (asdetermined from physiological monitoring data) during the run. Or, thedisplay of position information during an audiovisual recording of askier's activity can aid a viewer's recollection of where on a mountainparticular skiing occurred.

A recording can also be displayed in a variety of ways as a result ofparticular processing of the recording data. For example, the displayrate of a recording can be varied (either speeded up or slowed down). Ingeneral, any appropriate method can be used to vary the display rate ofa recording obtained by a recording unit according to the invention. Forexample, a method as described in the above-referenced U.S. patentapplication Ser. No. 08/760,769 can be used to vary the display rate ofa recording.

A recording can also be summarized. In general, any appropriatesummarization method can be used to summarize a visual recording and/oran audio recording obtained by a recording unit according to theinvention. For example, a method as described in commonly owned,co-pending U.S. patent application Ser. No. 08/761,025, entitled“Indirect Manipulation of Data Using Temporally Related Data, withParticular Application to Manipulation of Audio or Audiovisual Data,”filed on Dec. 5, 1996, by Emanuel E. Farber and Subutai Ahmad, thedisclosure of which is incorporated by reference herein, can be used tosummarize a recording.

It may also be desirable to extract keyframes from a visual recording.In general, this can be done using any appropriate method. For example,a method as described in commonly owned, co-pending U.S. patentapplication Ser. No. 08/528,891, entitled “A Method of Compressing aPlurality of Video Images for Efficiently Storing, Displaying andSearching the Plurality of Video Images,” filed on Sep. 15, 1995, bySubutai Ahmad, the disclosure of which is incorporated by referenceherein, can be used to extract keyframes from a visual recording.

2. Visual Recording Display

When the visual recording is a continuous visual recording of the eventobtained by a video camera, the display of the visual recording isusually a display of all, or one or more segments, of the visualrecording. Such display can be generated using any appropriateapparatus, such as a computer display monitor or television. Stillimages can be displayed on a display device such as a computer displaymonitor or television, or can be displayed as photographs.

Simultaneous display of multiple visual recordings can be accomplishedby providing a visual display device for each visual recording.Alternatively, it can be desirable to provide a single visual displaydevice having the capability to simultaneously display multiple visualrecordings. With the addition of appropriate processing capability(e.g., specialized video cards), computer display monitors can displaymultiple visual recordings at one time. Televisions also havecapabilities such as picture-in-picture that allow simultaneous displayof multiple recordings.

Multiple visual recordings can be combined to form a panoramic visualdisplay (either a video display or a still image). To construct such apanoramic visual display, the spatial relationship of the content of thevisual recordings must be determined, as discussed above. Overlappingand adjacent images of different recordings that were obtained at thesame time can be blended (using methods known to those skilled in theart; see, e.g., the first and second Szeliski publications referencedabove, and the paper entitled “Dealing with Speed and Robustness Issuesfor Video-Based Registration on a Wearable Computing Platform,” by Li-TeCheng et al., IEEE Second International Symposium on Wearable Computers,Oct. 19-20, 1998, p. 84 et seq., referred to hereinafter as the “Chengpublication”, the disclosure of which is incorporated by referenceherein); or otherwise combined to form a single image.

Multiple visual recordings can also be combined to form a visual display(either a video display or a still) other than a panoramic display(i.e., the combined images need not be from the same nodal point). Theimages can be combined to form either two-dimensional orthree-dimensional displays. Overlapping images can be combined and/orimages can be synthesized to fill in gaps between images generated fromrecording data. Techniques that can be used to generate such visualdisplays are described in, for example, the paper entitled, “AnArchitecture for Multiple Perspective Viewing,” by Patrick Kelly et al.,ACM Multimedia '95, 1995, San Francisco, Calif., pp. 201-212, thedisclosure of which is incorporated by reference herein.

Multiple visual recordings can also be displayed simultaneously in anon-integrated way. For example, multiple visual recordings can bedisplayed together to form a visual collage (with or without audio). Thecollage can be comprised of moving or still visual images.

Though a continuous visual recording can (and often will) be displayedby displaying all, or one or more segments, of the visual recording, itis also possible to display only single images (e.g., frames) of thevisual recording, thus effectively producing a display similar to thatproduced by a visual recording comprised of a set of still images. Whilethe displays are similar, it can be preferable to display single imagesfrom, or based upon, a continuous visual recording for a variety ofreasons. For example, a continuous visual recording provides a largerset of images from which to select an image to be displayed.

It is also possible to synthesize a single image from multiple images ofa continuous visual recording (see, e.g., the second Mann publicationreferenced above). Such synthesis can be used to generally improve thequality of a continuous visual recording or to produce very high-qualitystill images.

Single images from a continuous visual recording can also be used tocreate a panoramic image. (In fact, the recording unit can intentionallybe operated, i.e., the visual data acquisition device can be panned overa desired area—in either a controlled or casual manner—by the recorderso as to acquire single images that enable the creation of such apanoramic image.) Images of adjacent and/or overlapping content can bestitched together using one of a variety of known techniques to createthe panoramic image. Such stitching methods that can be used with theinvention are described in, for example, U.S. Pat. Nos. 5,396,583,5,510,830 and 5,262,856, the disclosures of which are incorporated byreference herein. Methods that can be used are also described in thefirst and second Szeliski publications referenced above, as well as theCheng publication referenced above. If the content of the scene of whichit is desired to create a panoramic image is relatively unchanging,differences in time of acquisition of the images used to create thepanoramic image are often relatively unimportant. As the rapidity withwhich the content of the scene changes increases, it becomes relativelymore important to obtain the images within a short time period of eachother.

Single images from a continuous visual recording can also be used tocreate other types of composite images. There are a large variety oftypes of composite images and techniques for creating composite imagesthat have been developed by others. In general, any of those techniquescan be used to create a composite image, and any of those types ofcomposite images can be created, from the images of a continuous visualrecording obtained by a recording unit according to the invention. Forexample, some methods enable the visual recording data for a visualrecording of an event of known type to be analyzed and images ofactivity meeting certain criteria culled from the visual recording andcombined into a single visual display (e.g., a collage). (Such methodscan also be used to summarize a recording and/or to enable searching ofa recording, as can be appreciated by those skilled in the art.)Illustratively, a method of this type can enable images of a run beingscored to be extracted from a visual recording of a baseball game.

3. Audio Recording Display

In general, the audio recording display can be generated using anyappropriate devices. Often, the audio recording display will begenerated using speakers that are part of a device used to generate thevisual recording display.

Audio recording data can be processed so that only particular audiocontent from the audio recording is displayed. For example, as describedabove, beam forming can be used to discriminate among various soundsources in an audio recording. Such discrimination can be used to causerecording content to be displayed that includes sounds emanating from aparticular source or sources (e.g., a particular person or group ofpersons).

As described above, it is desirable to display multiple recordingssimultaneously. For such simultaneous display, the audio portion ofmultiple recordings can be integrated to produce a single audiorecording display or the audio recordings can be displayedsimultaneously without integration.

It may be possible and desirable to integrate audio recordings obtainedby different audio data acquisition devices by blending the audiorecording data acquired by those audio data acquisition devices. It maybe particularly desirable—and most feasible—to blend audio recordingsobtained by multiple data acquisition devices that are part of the samerecording unit. It may also be possible to blend audio recordingsobtained by multiple data acquisition devices that are part of thedifferent recording units. However, in the latter case, during some orall of an event recorders may be sufficiently far removed from eachother that their recording units acquire audio recordings includingaudio content that is so different that the audio content cannot beblended in a meaningful way.

Audio recording data can be blended in any appropriate way. For example,an audio mixer can be used to select audio recording data from multipleaudio recordings in accordance with specified characteristics, and theselected audio recording data used to generate an audio display. Or, forexample, position information obtained during recording can be used toidentify the location at which particular audio recording data wasobtained and the audio recording data mixed as appropriate. Or, forexample, as described above, position information obtained duringrecording can be used with a beam forming technique to isolateparticular sounds in the audio. The audio recordings of sounds to beretained in the audio display are blended and displayed.

It can also be desirable to simultaneously display multiple audiorecordings without blending the audio recording data. Omni-directionalmicrophones that can be used in a recording unit according to theinvention to acquire audio data can overly emphasize the loudness of therecorder's voice. Directional microphones are intended to acquire audiodata preferentially; it may be realized after the event has beenrecorded that such preferential recording has caused sounds of interestto be minimized or eliminated from the audio recording. Displayingmultiple recordings can ameliorate both of these problems: in the formercase, the addition of other recordings reduces the prominence of arecorder's voice from any one recording, while in the latter, theaddition of other recordings increases the chance that audio content notcaptured satisfactorily in one recording will be recorded adequately inanother. Further, during an event, recorders may be sufficiently farremoved from each other that, apart from any limitations of theparticular audio data acquisition devices used, the recording unitsacquire audio recordings including different audio content. Simultaneousdisplay of audio recordings can give a more complete representation ofthe audio content over the space traversed by recorders during an event,both by better reproducing the content that existed and by betterreplicating the richness of the audio space (particularly if the audiodisplay is implemented so as to reproduce the spatial relationships ofthe recording units during recording) that may have existed in thatspace.

Various embodiments of the invention have been described. Thedescriptions are intended to be illustrative, not limitative. Thus, itwill be apparent to one skilled in the art that certain modificationsmay be made to the invention as described above without departing fromthe scope of the claims set out below.

We claim:
 1. A hybrid recording unit for use in recording an event,comprising: a portable video recorder, the portable video recorderconfigured to receive data from a portable video camera configured togenerate video data representing the event, wearable by a user, theportable video recorder, further comprising: means for recording livesaid video data representing the event; and an external communicationport for communicating with an external device; an auxiliary device,comprising: means for producing and/or acquiring auxiliary data duringthe recording of the event; and means for communicating with theportable video recorder via the external communication port of theportable video recorder; and means for establishing a temporalcorrespondence between the auxiliary data and: said recorded video datarepresenting the event; a controller configured to monitor aphysiological characteristic of the user wherein the physiologicalcharacteristic identifies a part of the event and may be used to modifythe recording of the event; and wherein establishing a temporalcorrespondence comprises associating the auxiliary data with acorresponding portion of the recorded video data representing the eventthat was recorded at approximately the same time that the auxiliary datawas produced and/or acquired.
 2. A hybrid recording unit as in claim 1,wherein the auxiliary data is of a type that cannot be acquired by theportable video recorder.
 3. A hybrid recording unit as in claim 1,wherein the means for producing and/or acquiring auxiliary data furthercomprises means for specifying a mark that is associated with particularrecording data acquired by the portable video recorder.
 4. A hybridrecording unit as in claim 3, wherein the means for specifying a markfurther comprises means for specifying multiple types of marks.
 5. Ahybrid recording unit as in claim 1, wherein the auxiliary devicefurther comprises a non-volatile data storage device for storingauxiliary data.
 6. A hybrid recording unit as in claim 5, wherein thenon-volatile data storage device further comprises an EEPROM.
 7. Ahybrid recording unit as in claim 5, wherein the non-volatile datastorage device further comprises a flash memory device.
 8. An auxiliarydevice for use with a portable video recorder, the portable videorecorder configured to receive data from a portable video cameraconfigured to generate video data representing an event, the portablevideo recorder, wearable by a user, further being adapted to record livesaid video data representing the event, the auxiliary device comprising:means for producing and/or acquiring auxiliary data during said liverecording of the event; means for communicating with the portable videorecorder via an external communication port of the portable videorecorder; a non-volatile data storage device for storing auxiliary data;and means for establishing a temporal correspondence between theauxiliary data and said recorded video data representing the event; acontroller configured to monitor a physiological characteristic of theuser wherein the physiological characteristic identifies a part of theevent and may be used to modify the recording of the event; and whereinestablishing a temporal correspondence comprises associating theauxiliary data with a corresponding portion of the recorded video datarepresenting the event that was recorded at approximately the same timethat the auxiliary data was produced and/or acquired.